This content originally appeared on Level Up Coding - Medium and was authored by Jarek Orzel
Imagine what happens under the hood when you run a command like redis-cli GET x from your application. How does that simple text command translate into storage, responses, and internal logic?
In this blog series, I want to walk you through the process of designing and building a Redis-like server from scratch, starting with a simplified single-node implementation in Golang. Future posts will delve deeper into topics such as replication, transactions, and persistence.
⚠️ Disclaimer: This is not an official Redis implementation. These posts are part of my personal learning journey — an attempt to better understand how key-value databases work internally. The design will be simplified, but the goal is to gain insight into the architecture and core mechanics that power systems like Redis.
So with that said, let’s start from the beginning: building a basic event loop, parsing commands, handling requests, and storing data in memory.
Architecture
In this post, I’ll assume a single-server setup that handles both read and write commands.
The architecture of this simplified Redis-like system can be broken down into the following core components.
Connection Listener
The connection listener is the entry point for all client communication. It listens for incoming TCP connections (e.g., on port 6379). For each accepted connection, it hands it off to the connection handler. Its role is minimal but essential.
The connection listener is designed to manage many client connections concurrently, ensuring efficient and scalable access to the server.
Connection Handler
The connection handler is responsible for orchestrating the full request-response lifecycle for each client connection. It repeatedly reads data from the client, invokes the command parser to interpret the request, dispatches the parsed command to the appropriate command handler, and finally sends the response back using the response serializer.
Command Parser
Redis uses a specific wire protocol called RESP (REdis Serialization Protocol) for both input and output communication. Every official Redis client implements RESP to talk to the server, making it a core part of the ecosystem.
RESP is fast, simple, and designed for high-performance command parsing. Redis typically uses RESP as a request-response protocol, where:
- Clients send commands as an array of bulk strings.
- The first string is the command name (e.g. "SET", "GET").
- Remaining elements are treated as arguments (e.g. key and value).
In RESP, the first byte of a message determines the data type:
- For Simple Strings, the first byte of the reply is “+"
- For Errors, the first byte of the reply is “-"
- For Integers, the first byte of the reply is “:"
- For Bulk Strings, the first byte of the reply is “$"
- For Arrays, the first byte of the reply is “*"
A client command like:
SET x 123
is sent over the wire as an array of bulk strings:
*3\r\n$3\r\nSET\r\n$1\r\nx\r\n$3\r\n123\r\n
*<value> represents the number of elements in the array, while each $<value> determines the length of the given element.
The parser reads this raw input and transforms it into a structured array:
[]string{"SET", "x", "123"}This parsed structure is then passed to the command dispatcher for execution.
Command Handler
The command handler implements the logic for each supported Redis command like SET, GET, PING, etc. After parsing, the dispatcher routes the command to the appropriate handler based on its name. Each handler may access or update the in-memory store and return a result to be serialized as a response.
This layer is the business logic core of your database.
Storage
The storage component is an in-memory key-value store (typically backed by a map or hash table). It holds the current state of the database. Command handlers interact with this store to read and modify values.
It is intentionally simple in a single-node architecture, but can later evolve to include TTL expiration, eviction policies, or persistence.
Response Serializer
The response serializer performs the opposite of parsing: it takes the result of command execution and encodes it into a valid RESP response to send back to the client.
Whether it’s a simple acknowledgment (OK), a string value, or an error message — every outgoing response must follow RESP conventions so that clients can correctly interpret it.
serialize("+OK") // returns "+OK\r\n"
serialize("value") // returns "$5\r\nvalue\r\n"
serialize(nil) // returns "$-1\r\n"
serialize([]string{"a", "b"}) // returns "*2\r\n$1\r\na\r\n$1\r\nb\r\n"This ensures full compatibility with Redis clients and predictable behavior across command types.
Implementation
With the architecture in place and a basic understanding of how Redis uses the RESP protocol for communication, we’re ready to dive into the actual implementation. We’ll start by building out the essential components, beginning with the connection listener and handler, then move through the command parser, dispatcher, response serializer, and finally the in-memory storage engine. Each piece will be kept simple but functional, focusing on clarity and correctness over full feature parity with Redis.
⚠️ Suggestion:
This post dives into the actual implementation details of our Redis-like server. If you’re not interested in the code-level internals at this point, feel free to skip this section and jump to demonstration section.
Master Server
At this stage, we implement a simplified version of the MasterServer to work as a Redis single-node server.
package server
type MasterServer struct {
listener net.Listener
commandParser protocol.CommandParser
commandHandler commands.CommandHandler
config *config.Config
}
func NewMasterServer(config *config.Config) (*MasterServer, error) {
addr := fmt.Sprintf("0.0.0.0:%d", config.ServerPort)
ln, err := net.Listen("tcp", addr)
if err != nil {
return nil, err
}
return &MasterServer{
listener: ln,
commandParser: protocol.NewCommandParser(),
commandHandler: commands.NewCommandHandler(config),
config: config,
}, nil
}
func (ms *MasterServer) Start(ctx context.Context) error {
logger := zerolog.Ctx(ctx)
logger.Info().Str("address", ms.listener.Addr().String()).Msg("Master listening on...")
for {
conn, err := ms.listener.Accept()
if err != nil {
continue
}
go ms.handleConnection(ctx, conn)
}
}
🔧 Key Components:
- net.Listener is used to bind to a TCP port and accept new client connections.
- handleConnection is launched as a goroutine for each connection to provide concurrent access to the server.
- CommandParser and CommandHandler are injected and delegated to parse and handle RESP commands from clients.
Connection Handler
The handleConnection method is the core loop for processing incoming client commands. While in a more modular design, this might be a standalone component, here it's embedded directly within the server for simplicity.
package server
func (ms *MasterServer) handleConnection(ctx context.Context, conn net.Conn) {
defer conn.Close()
logger := zerolog.Ctx(ctx).With().
Str("remote_addr", conn.RemoteAddr().String()).
Logger()
logger.Info().Msg("Handling new connection")
buffer_size := 1024*4 // 4 KB
buffer := make([]byte, buffer_size)
for {
n, err := conn.Read(buffer)
if err != nil {
if err == io.EOF {
logger.Info().Msg("Connection closed by client")
return
}
logger.Err(err).Msg("Error reading from connection")
return
}
logger.Debug().Str("data", string(buffer[:n])).Msg("Received data")
result, err := ms.commandParser.Parse(buffer[:n])
if err != nil {
logger.Err(err).Msg("Failed to parse received data")
continue
}
for _, command := range result.Commands {
logger := logger.With().
Str("command", command.Name).
Interface("args", command.Args).Logger()
logger.Info().Msg("Parsed command")
result, err := ms.commandHandler.Handle(ctx, conn, command)
if err != nil {
logger.Err(err).Msg("Failed to handle command")
continue
}
if result.CommandError != nil {
logger.Err(result.CommandError).Msg("Command error occurred, sending error response")
}
}
}
}
🔍 Step-by-step Explanation:
- We allocate a 4KB buffer and read raw bytes directly from the TCP stream using conn.Read.
- Instead of using bufio.Reader in ms.commandParser, we directly read into the buffer to make debugging easier at this stage. This way, you can log exactly what was received — useful when learning how RESP messages are structured.
- The received data is passed to ms.commandParser.Parse(...). Our parser must be capable of splitting and identifying multiple RESP commands in one chunk, which is a realistic scenario in Redis where pipelined commands can arrive batched.
- Each parsed command is passed to ms.commandHandler.Handle(...), which executes it and (optionally) returns a response.
Command Parser
Our CommandParser interprets RESP requests and transforms them into structured command data (Command object).
At this stage, it supports basic RESP arrays made up of bulk strings — the format used by almost every Redis command.
package protocol
type Command struct {
Name string
Args []string
}
type ParseResult struct {
Commands []Command
}
type DefaultCommandParser struct{}
func (p DefaultCommandParser) Parse(rawMessage []byte) (ParseResult, error) {
result := ParseResult{}
reader := bufio.NewReader(bytes.NewReader(rawMessage))
for {
b, err := reader.Peek(1)
if err != nil {
return result, err
}
switch b[0] {
case '*':
cmd, err := readCommand(reader)
if err != nil {
return result, err
}
result.Commands = append(result.Commands, cmd)
default:
return result, fmt.Errorf("unsupported RESP message start byte: %q", b[0])
}
}
return result, nil
}
func readCommand(r *bufio.Reader) (Command, error) {
line, err := readLine(r)
if err != nil {
return Command{}, err
}
if len(line) == 0 || line[0] != '*' {
return Command{}, fmt.Errorf("expected array, got: %q", line)
}
argCount, err := strconv.Atoi(line[1:])
if err != nil {
return Command{}, fmt.Errorf("invalid array length: %w", err)
}
args := make([]string, 0, argCount)
for i := 0; i < argCount; i++ {
arg, err := readBulkString(r)
if err != nil {
return Command{}, err
}
args = append(args, arg)
}
if len(args) == 0 {
return Command{}, fmt.Errorf("empty command")
}
return NewCommand(args[0], args[1:]), nil
}
func readBulkString(r *bufio.Reader) (string, error) {
line, err := readLine(r)
if err != nil {
return "", err
}
if len(line) == 0 || line[0] != '$' {
return "", fmt.Errorf("expected bulk string, got: %q", line)
}
length, err := strconv.Atoi(line[1:])
if err != nil {
return "", fmt.Errorf("invalid bulk length: %w", err)
}
buf := make([]byte, length)
if _, err := io.ReadFull(r, buf); err != nil {
return "", err
}
crlf := make([]byte, 2)
if _, err := io.ReadFull(r, crlf); err != nil {
return "", err
}
if !bytes.Equal(crlf, []byte(CRLF)) {
return "", fmt.Errorf("expected CRLF after bulk string, got: %q", crlf)
}
return string(buf), nil
}
func readLine(r *bufio.Reader) (string, error) {
line, err := r.ReadString('\n')
if err != nil {
return "", err
}
if len(line) < 2 || line[len(line)-2] != '\r' {
return "", fmt.Errorf("protocol error: expected CRLF ending, got %q", line)
}
return line[:len(line)-2], nil // strip \r\n
}
How the Parser Works:
- Parse(rawMessage []byte): Currently only supports RESP arrays (*) with bulk strings ($) — enough to cover most real-world commands like SET, GET, PING, etc.
- readCommand(...): Reads the array size (number of arguments) from a line like *3\r\n. Then repeatedly calls readBulkString(...) to extract each argument, such as $3\r\nSET\r\n.
- The result of Parseis a list of Command objects.
Command Handler
At this point in our design, the Command Handler plays a dual role — it’s responsible for both dispatching and executing commands. It acts like a router, directing different Redis commands (GET, SET, PING, DEL, etc.) to their specific logic implementations.
const CRLF = `\r\n`
type HandleResult struct {
CommandError error
}
type CommandHandler interface {
Handle(ctx context.Context, conn net.Conn, command protocol.Command) (HandleResult, error)
}
type DefaultCommandHandler struct {
storage storage.Storage
config *config.Config
}
func (h *DefaultCommandHandler) Handle(
ctx context.Context, conn net.Conn, command protocol.Command,
) (HandleResult, error) {
switch command.Name {
case protocol.PING:
return h.handlePing(ctx, conn, command)
case protocol.SET:
return h.handleSet(ctx, conn, command)
case protocol.GET:
return h.handleGet(ctx, conn, command)
case protocol.DEL:
return h.handleDel(ctx, conn, command)
default:
...
}
}
The specific implementations of each handler can be found in the repo (link at the end of the post). Now we show only how GET is implemented.
package commands
func (h *DefaultCommandHandler) handleGet(
ctx context.Context, conn net.Conn, command protocol.Command,
) (HandleResult, error) {
msg, commandErr := h.executeGet(ctx, command)
err := h.sendMsg(ctx, conn, msg)
return HandleResult{
CommandError: commandErr,
}, err
}
func (h *DefaultCommandHandler) executeGet(_ context.Context, command protocol.Command) ([]byte, error) {
if len(command.Args) != 1 {
errMsg := "GET command requires exactly 1 argument"
return protocol.Error(errMsg), fmt.Errorf(errMsg)
}
deserializedRecord, err := h.storage.Get(command.Args[0])
if err != nil {
errMsg := "Failed to get record: " + err.Error()
return protocol.Error(errMsg), fmt.Errorf(errMsg)
}
if deserializedRecord == nil {
return protocol.Nil(), nil
}
return protocol.BulkString(deserializedRecord.Value), nil
}
GET handler checks whether the key can be found in the storage.
- If the key doesn’t exist, it returns a nil response (encoded in RESP as $-1\r\n).
- If the key is found and valid, it returns the associated value as a bulk string.
- The response value is encoded using the protocol serializer.
Serializer
All response commands are defined in the serialization protocol module. Some examples:
package protocol
const CRLF = `\r\n`
func SimpleString(s string) []byte {
return []byte("+" + s + CRLF)
}
func Error(s string) []byte {
return []byte("-ERR " + s + CRLF)
}
func BulkString(s string) []byte {
return []byte("$" + strconv.Itoa(len(s)) + CRLF + s + CRLF)
}
func Nil() []byte {
return []byte("$-1" + CRLF)
}
func BulkArray(elements []string) []byte {
result := "*" + strconv.Itoa(len(elements)) + CRLF
for _, element := range elements {
result += string(BulkString(element))
}
return []byte(result)
}
Storage
Our storage layer is intentionally simple at this stage — it’s a thin wrapper around Go’ssync.Map, which gives us safe concurrent access out of the box.
This is the foundation for the in-memory part of our Redis-like server — all GET, SET, and DEL commands interact with this component.
package storage
type KVRecord struct {
Value string
ExpireAt *time.Time // for future EX/PX implementation
}
type DefaultStorage struct {
db sync.Map
}
func (s *DefaultStorage) Get(key string) (*KVRecord, error) {
record, ok := s.db.Load(key)
if !ok {
return nil, nil
}
deserializedRecord, ok := record.(*KVRecord)
if !ok {
return nil, fmt.Errorf("failed to deserialize record for key%s, record=%v", key, record)
}
return deserializedRecord, nil
}
func (s *DefaultStorage) Set(key string, value *KVRecord) error {
s.db.Store(key, value)
return nil
}
func (s *DefaultStorage) Del(key string) error {
if _, ok := s.db.Load(key); !ok {
return fmt.Errorf("key %s does not exist", key)
}
s.db.Delete(key)
return nil
}
Main Runner
The main.go file serves as the entry point and ties all components together.
package main
func main() {
logger := zerolog.New(os.Stderr).With().Timestamp().Logger()
zerolog.TimeFieldFormat = zerolog.TimeFormatUnixMs
zerolog.SetGlobalLevel(zerolog.DebugLevel)
ctx := logger.WithContext(context.Background())
initSpec, err := getInitSpecsFromArgs()
if err != nil {
logger.Err(err).Msg("Failed to parse initialization specifications from arguments")
os.Exit(1)
}
config, err := config.NewConfig(initSpec)
if err != nil {
logger.Err(err).Msg("Failed to create configuration")
os.Exit(1)
}
logger.Info().Interface("config", config).Msg("Starting configuration")
srv, err := server.NewMasterServer(config)
if err != nil {
logger.Err(err).Msg("Failed to initialize server")
os.Exit(1)
}
err = srv.Start(ctx)
if err != nil {
logger.Err(err).Msg("Failed to start server")
os.Exit(1)
}
}
func getInitSpecsFromArgs() (config.InitSpec, error) {
port := flag.Int("port", 6379, "Port to listen on")
flag.Parse()
if port == nil {
return config.InitSpec{}, fmt.Errorf("port flag is required")
}
if *port < 1 || *port > 65535 {
return config.InitSpec{}, fmt.Errorf("port must be between 1 and 65535, got %d", *port)
}
flag.Parse()
return config.InitSpec{
ServerPort: *port,
}, nil
}
The main function:
- Initializes logging and context.
- Parses startup flags (like --port).
- Loads the configuration.
- Initializes and runs the Redis-like server.
Now we are ready to test our implementation.
Demonstration
To demonstrate our solution, we’ve prepared two helper scripts: compile.sh and run.sh. The first one builds the server, and the second launches it. To interact with the server and test how it handles commands, we’ll use the standard Redis CLI. If you don’t already have the redis-cli tool installed, you can follow the official instructions here.
Once installed, you can use it like this:
$ redis-cli SET key value
$ redis-cli GET key
This lets us verify how our custom server responds to real Redis commands.
$ ./compile.sh
$ ./run.sh
{"level":"info","config":{"port":6379},"time":1749553784033,"message":"Starting configuration"}
{"level":"info","address":"[::]:6379","time":1749553784033,"message":"Master listening on..."}
Our server started and is ready to accept requests from clients. Our server started on the default 6379 port. However, we can customize it by passing —-port flag (e.g. ./run.sh --port 6890 ).
In a separate terminal, we run several commands:
$ redis-cli PING
PONG
$ redis-cli GET a
(nil)
$ redis-cli SET a 1
OK
$ redis-cli GET a
"1"
$ redis-cli DEL a
(integer) 1
$ redis-cli GET a
(nil)
These responses show that our server correctly implements these four basic commands. From the server side, we can also check logs:
{"level":"info","config":{"port":6379},"time":1749553784033,"message":"Starting configuration"}
{"level":"info","address":"[::]:6379","time":1749553784033,"message":"Master listening on..."}
{"level":"info","remote_addr":"127.0.0.1:54431","time":1749554012110,"message":"Handling new connection"}
{"level":"debug","remote_addr":"127.0.0.1:54431","data":"*1\r\n$4\r\nPING\r\n","time":1749554012110,"message":"Received data"}
{"level":"info","remote_addr":"127.0.0.1:54431","command":"PING","args":[],"time":1749554012110,"message":"Parsed command"}
{"level":"info","remote_addr":"127.0.0.1:54431","time":1749554012110,"message":"Connection closed by client"}
{"level":"info","remote_addr":"127.0.0.1:54434","time":1749554017169,"message":"Handling new connection"}
{"level":"debug","remote_addr":"127.0.0.1:54434","data":"*2\r\n$3\r\nGET\r\n$1\r\na\r\n","time":1749554017169,"message":"Received data"}
{"level":"info","remote_addr":"127.0.0.1:54434","command":"GET","args":["a"],"time":1749554017169,"message":"Parsed command"}
{"level":"info","remote_addr":"127.0.0.1:54434","time":1749554017169,"message":"Connection closed by client"}
{"level":"info","remote_addr":"127.0.0.1:54435","time":1749554020739,"message":"Handling new connection"}
{"level":"debug","remote_addr":"127.0.0.1:54435","data":"*3\r\n$3\r\nSET\r\n$1\r\na\r\n$1\r\n1\r\n","time":1749554020739,"message":"Received data"}
{"level":"info","remote_addr":"127.0.0.1:54435","command":"SET","args":["a","1"],"time":1749554020739,"message":"Parsed command"}
{"level":"info","remote_addr":"127.0.0.1:54435","time":1749554020739,"message":"Connection closed by client"}
{"level":"info","remote_addr":"127.0.0.1:54436","time":1749554022248,"message":"Handling new connection"}
{"level":"debug","remote_addr":"127.0.0.1:54436","data":"*2\r\n$3\r\nGET\r\n$1\r\na\r\n","time":1749554022248,"message":"Received data"}
{"level":"info","remote_addr":"127.0.0.1:54436","command":"GET","args":["a"],"time":1749554022248,"message":"Parsed command"}
{"level":"info","remote_addr":"127.0.0.1:54436","time":1749554022249,"message":"Connection closed by client"}
{"level":"info","remote_addr":"127.0.0.1:54438","time":1749554025665,"message":"Handling new connection"}
{"level":"debug","remote_addr":"127.0.0.1:54438","data":"*2\r\n$3\r\nDEL\r\n$1\r\na\r\n","time":1749554025665,"message":"Received data"}
{"level":"info","remote_addr":"127.0.0.1:54438","command":"DEL","args":["a"],"time":1749554025665,"message":"Parsed command"}
{"level":"info","remote_addr":"127.0.0.1:54438","time":1749554025665,"message":"Connection closed by client"}
{"level":"info","remote_addr":"127.0.0.1:54439","time":1749554027959,"message":"Handling new connection"}
{"level":"debug","remote_addr":"127.0.0.1:54439","data":"*2\r\n$3\r\nGET\r\n$1\r\na\r\n","time":1749554027959,"message":"Received data"}
{"level":"info","remote_addr":"127.0.0.1:54439","command":"GET","args":["a"],"time":1749554027959,"message":"Parsed command"}
{"level":"info","remote_addr":"127.0.0.1:54439","time":1749554027959,"message":"Connection closed by client"}Conclusion
In this post, we explored the foundational architecture of a Redis-like key-value store. We introduced the core components — including the connection listener, command parser, dispatcher, response serializer, and storage — and implemented several basic commands like PING, GET, SET, and DEL.
This simplified implementation is just the beginning. In the next post, we’ll take things further by transforming our single-node server into a distributed system, introducing support for a master node with multiple replicas, along with command propagation and synchronization.
Stay tuned — the journey into Redis internals is just getting started.
🔗 Full code
The entire codebase for this Redis-like server is available on GitHub. Feel free to explore: https://github.com/jorzel/myredis
References:
- https://build-your-own.org/database/
- https://codingchallenges.fyi/challenges/challenge-redis/#step-zero
- https://www.build-redis-from-scratch.dev/en/introduction
- https://app.codecrafters.io/courses/redis/
Building Your Redis From Scratch: Implementing a Basic Server in Go was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Jarek Orzel
Jarek Orzel | Sciencx (2025-06-11T15:05:21+00:00) Building Your Redis From Scratch: Implementing a Basic Server in Go. Retrieved from https://www.scien.cx/2025/06/11/building-your-redis-from-scratch-implementing-a-basic-server-in-go/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.