This content originally appeared on Level Up Coding - Medium and was authored by Adam Szpilewicz
Understanding Byte Order and Its Practical Applications in Golang
Little Endian and Big Endian are two widely used byte orders in computer systems, which dictate how data is stored and accessed in memory. Understanding these concepts is crucial for developers, especially when dealing with low-level programming, binary file formats, or communication protocols. In this article, we’ll discuss the differences between Little Endian and Big Endian, their use cases, and provide examples using the Go programming language (Golang).
Little Endian vs. Big Endian
Endian refers to the order in which bytes are stored for multi-byte data types such as integers or floating-point numbers. There are two primary ways to represent these values in memory:
Little Endian: The least significant byte (LSB) is stored at the lowest memory address, with the remaining bytes stored in increasing order of significance. For example, the 32-bit integer 0x12345678 would be stored as 0x78, 0x56, 0x34, 0x12.
Let’s consider the example of the 32-bit integer 0x12345678:
In hexadecimal notation, each digit represents 4 bits. Therefore, a 32-bit integer has 8 hexadecimal digits. In this case, the 8 hexadecimal digits are: 12, 34, 56, and 78.
When the integer 0x12345678 is stored in memory using Little Endian byte order, the least significant byte (0x78) is stored first, at the lowest memory address. The remaining bytes are stored in increasing order of significance, like this:
Memory Address: | ... | n | n+1 | n+2 | n+3 | ... |
-------------------------------------
Byte Value: | ... | 0x78 | 0x56 | 0x34 | 0x12 | ... |
In this layout, the memory address increases from left to right. The least significant byte (0x78) is stored at the lowest memory address n, followed by the other bytes in increasing order of significance: 0x56 at address n+1, 0x34 at address n+2, and finally the most significant byte (0x12) at address n+3.
Big Endian: The most significant byte (MSB) is stored at the lowest memory address, with the remaining bytes stored in decreasing order of significance. In this case, the integer 0x12345678 would be stored as 0x12, 0x34, 0x56, 0x78.
Let’s consider the example of the 32-bit integer 0x12345678:
In hexadecimal notation, each digit represents 4 bits. Therefore, a 32-bit integer has 8 hexadecimal digits. In this case, the 8 hexadecimal digits are: 12, 34, 56, and 78.
When the integer 0x12345678 is stored in memory using Big Endian byte order, the most significant byte (0x12) is stored first, at the lowest memory address. The remaining bytes are stored in decreasing order of significance, like this:
Memory Address: | ... | n | n+1 | n+2 | n+3 | ... |
-------------------------------------
Byte Value: | ... | 0x12 | 0x34 | 0x56 | 0x78 | ... |
In this layout, the memory address increases from left to right. The most significant byte (0x12) is stored at the lowest memory address n, followed by the other bytes in decreasing order of significance: 0x34 at address n+1, 0x56 at address n+2, and finally the least significant byte (0x78) at address n+3.
Which one is more readable to humans
Big Endian notation is generally considered more human-readable than Little Endian notation. The reason for this is that Big Endian byte order stores the most significant byte (MSB) at the lowest memory address and the remaining bytes in decreasing order of significance. This representation aligns with how we read and write numbers in positional numeral systems, such as the decimal system, from left to right with the most significant digit first.
For example, consider the 32-bit hexadecimal integer 0x12345678. In Big Endian byte order, the integer is stored as 0x12, 0x34, 0x56, 0x78. This is the same order in which we naturally read and write the number, so it is easier for humans to interpret and understand.
In contrast, Little Endian notation stores the least significant byte (LSB) at the lowest memory address and the remaining bytes in increasing order of significance. Using the same example, the integer 0x12345678 would be stored as 0x78, 0x56, 0x34, 0x12 in Little Endian byte order. This reversed order can be more challenging for humans to read and understand since it does not align with the way we typically read and write numbers.
However, it is important to note that the choice between Little Endian and Big Endian conventions is primarily a technical concern for computer systems, as they handle the storage and processing of binary data. For humans, the readability difference between the two notations is less significant, as we generally work with higher-level representations of numbers and not their byte-level storage formats.
Use Cases
Each byte order has its advantages and disadvantages, and the choice depends on the specific requirements of the system or application. Some considerations include:
- Little Endian: This byte order is commonly used in x86 and ARM processors, as well as Windows and Linux operating systems. Little Endian is generally preferred for mathematical operations, as it aligns with how humans perform arithmetic.
- Big Endian: Big Endian is often used in networking protocols, such as TCP/IP, where data is transmitted in a consistent and easily understood format. It is also used in some processor architectures, like the Motorola 68000 and IBM POWER series.
Golang Examples
The “encoding/binary” package in Go provides functions to read and write binary data using both Little Endian and Big Endian byte orders. Here are some examples to demonstrate these functions:
Writing an integer in Little Endian and Big Endian formats:
package main
import (
"bytes"
"encoding/binary"
"fmt"
)
func main() {
buf := new(bytes.Buffer)
var num int32 = 0x12345678
// Write as Little Endian
binary.Write(buf, binary.LittleEndian, num)
fmt.Printf("Little Endian: %X\n", buf.Bytes())
buf.Reset()
// Write as Big Endian
binary.Write(buf, binary.BigEndian, num)
fmt.Printf("Big Endian: %X\n", buf.Bytes())
}
Output:
Little Endian: 78563412
Big Endian: 12345678
Reading an integer from Little Endian and Big Endian data:
package main
import (
"bytes"
"encoding/binary"
"fmt"
)
func main() {
littleEndianData := []byte{0x78, 0x56, 0x34, 0x12}
bigEndianData := []byte{0x12, 0x34, 0x56, 0x78}
var littleEndianNum, bigEndianNum int32
// Read as Little Endian
binary.Read(bytes.NewReader(littleEndianData), binary.LittleEndian, &littleEndianNum)
fmt.Printf("Little Endian: %X\n", littleEndianNum)
// Read as Big Endian
binary.Read(bytes.NewReader(bigEndianData), binary.BigEndian, &bigEndianNum)
fmt.Printf("Big Endian: %X\n", bigEndianNum)
}
Output:
Little Endian: 12345678
Big Endian: 12345678
Both Little Endian and Big Endian data are read and converted to the same integer value 0x12345678.
My experience with Litte and Big Endian
As a backend developer working with Go, I encountered an issue while extracting data from a SQL Server table in a change data capture application (CDC). In SQL Server, the UNIQUEIDENTIFIER type is stored as LittleEndian. However, to properly parse that column we expected the raw data to be stored as BigEndian. To address this discrepancy, I needed to reorder the raw bytes extracted from sqlserver by our cdc application to ensure that the UUID values in both the source and target systems were the same. Here’s a code snippet that demonstrates the byte reordering process:
package main
import (
"encoding/binary"
"fmt"
"github.com/google/uuid"
)
func fromLittleEndianGUID(data []byte) (uuid.UUID, error) {
if len(data) != 16 {
return uuid.Nil, fmt.Errorf("input byte array should have a length of 16")
}
bigEndianData := make([]byte, 16)
binary.BigEndian.PutUint32(bigEndianData[:4], binary.LittleEndian.Uint32(data[:4]))
binary.BigEndian.PutUint16(bigEndianData[4:6], binary.LittleEndian.Uint16(data[4:6]))
binary.BigEndian.PutUint16(bigEndianData[6:8], binary.LittleEndian.Uint16(data[6:8]))
copy(bigEndianData[8:], data[8:])
return uuid.FromBytes(bigEndianData)
}
func main() {
littleEndianGUIDData := []byte{0x56, 0x69, 0xB9, 0x0C, 0xCF, 0xA2, 0xE9, 0x46, 0x9C, 0x69, 0x12, 0x34, 0x56, 0x78, 0x9A, 0xBC}
u, err := fromLittleEndianGUID(littleEndianGUIDData)
if err != nil {
fmt.Printf("Error parsing GUID: %v\n", err)
return
}
fmt.Printf("Parsed LittleEndian GUID: %s\n", u.String())
}
Parsed LittleEndian GUID: 0cb96956-a2cf-46e9-9c69-123456789abc
And the example of bytes reordering from go library dedicated to work with sqlserver:
https://github.com/denisenkom/go-mssqldb/blob/master/uniqueidentifier.go
Level Up Coding
Thanks for being a part of our community! Before you go:
- 👏 Clap for the story and follow the author 👉
- 📰 View more content in the Level Up Coding publication
- 💰 Free coding interview course ⇒ View Course
- 🔔 Follow us: Twitter | LinkedIn | Newsletter
🚀👉 Join the Level Up talent collective and find an amazing job
Little Endian and Big Endian was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Adam Szpilewicz
Adam Szpilewicz | Sciencx (2023-04-20T16:31:27+00:00) Little Endian and Big Endian. Retrieved from https://www.scien.cx/2023/04/20/little-endian-and-big-endian/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.