The internet has grown from a couple of research lab sites to a vibrant ecosystem of content and services. Without the ability to resolve human-readable names to machine identifiers, however, it’d be impossible to find anything in the ever-growing web of interconnected systems.
In November 1987, RFC 1034 and RFC 1035 first defined the domain name system (DNS), a distributed database containing the names of resources, designed to be extensible beyond the initial scope. Software including browsers and other internet-enabled apps refers to a DNS resolver, which queries name servers to resolve hostnames to IP addresses, mail settings, or other information.
To facilitate communication between DNS resolvers and name servers, RFC 1035 specifies the DNS wire format. DNS messages are transmitted as a series of octets (bytes). In this guide, we’ll try to understand what goes into a typical DNS query and response by implementing a DNS message parser in Go.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 1 | 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 3 | 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 5 | 6 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Before we get to parsing messages, let’s look at some fundamental concepts used through DNS, including domain names and resource records.
Domain Names
<domain> ::= <subdomain> | " "
<subdomain> ::= <label> | <subdomain> "." <label>
<label> ::= <letter> [ [ <ldh-str> ] <let-dig> ]
<ldh-str> ::= <let-dig-hyp> | <let-dig-hyp> <ldh-str>
<let-dig-hyp> ::= <let-dig> | "-"
<let-dig> ::= <letter> | <digit>
<letter> ::= any one of the 52 alphabetic characters A through Z in
upper case and a through z in lower case
<digit> ::= any one of the ten digits 0 through 9
Domain Names as defined in Section 2.3.1 and Section 3.1. are represented as sequences of labels separated by dots. Labels must start with letters followed by optional numbers and can contain, but not end with hyphens. Each label is represented as a one-octet length field followed by that number of octets.
Every domain name ends with the null label of the root (visualized using a trailing dot), which is why a domain name is terminated by a zero-length octet.
To accommodate for message compression, the first two high-order bits of every label’s length octet must be zero. In case compression is used, both bits are 1, followed by an offset value pointing to another label or domain name.
func parseDomainName(fullMessage []byte, bytes []byte) (string, []byte) {
domainName := ""
for {
length := bytes[0]
isCompressed := length&0b1100_0000 > 0
if isCompressed {
pointer := (bytes[0] << 2) + bytes[1]
resolvedPointer, _ := parseDomainName(fullMessage, fullMessage[pointer:])
domainName += resolvedPointer
bytes = bytes[2:]
break
}
bytes = bytes[1:]
if length == 0 {
break
}
label := bytes[0:length]
domainName += string(label)
domainName += "."
bytes = bytes[length:]
}
return domainName, bytes
}
Resource Records (RRs)
As I’ve said before, the domain name system was created as a database to translate resource names to information like IP addresses and other values. The atomic unit in this database is referred to as a Resource Record (RR) and is defined in Section 3.2.1. Each RR is addressed using a name (as defined above) and stores resource data (RDATA) for a given resource type (TYPE) and class (CLASS), expiring after the TTL.
1 1 1 1 1 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| |
/ /
/ NAME /
| |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| TYPE |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| CLASS |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| TTL |
| |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| RDLENGTH |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--|
/ RDATA /
/ /
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
Resource Records exist for different data types like IPv4 (A records) and IPv6 (defined in RFC 3596) addresses, as well as text (TXT) and mail-related (MX) records.
type RDataCNAME struct {
CName string // <domain-name> which specifies canonical or primary name for owner, owner name is alias
}
func (d RDataCNAME) String() string {
return d.CName
}
type RDataMX struct {
Preference uint16 // lower values are preferred
Exchange string // <domain-name> of host willing to act as mail exchange
}
func (d RDataMX) String() string {
return fmt.Sprintf("%d %s", d.Preference, d.Exchange)
}
type RDataNS struct {
NsdName string // <domain-name> specifying host which should be authoritative for specified class
}
func (d RDataNS) String() string {
return d.NsdName
}
type RDataTXT struct {
TxtData string // one or more character strings
}
func (d RDataTXT) String() string {
return d.TxtData
}
type RDataA struct {
Address string // host internet address
}
func (d RDataA) String() string {
return d.Address
}
type RDataAAAA struct { // RFC 3596
Address string // 128-bit IPv6 address
}
func (d RDataAAAA) String() string {
return d.Address
}
type ResourceRecord struct {
Name string // domain name associated with record
Type RecordType // two octets containing RR TYPE code
Class RecordClass // two octets containing RR class code
TTL uint32 // time in seconds until record should be refreshed in cache
RDLength uint16 // length of octets in RData
RData fmt.Stringer // variable-length string of octets describing resource, format varies by TYPE and CLASS
}
// transparently handles empty sections
func parseResourceRecords(fullMessage []byte, messageBytes []byte, numRecords uint16) ([]ResourceRecord, []byte, error) {
resourceRecords := make([]ResourceRecord, numRecords)
for i := range resourceRecords {
var domainName string
domainName, messageBytes = parseDomainName(fullMessage, messageBytes)
resourceRecords[i].Name = domainName
resourceRecords[i].Type = RecordType(binary.BigEndian.Uint16(messageBytes[0:2]))
messageBytes = messageBytes[2:]
resourceRecords[i].Class = RecordClass(binary.BigEndian.Uint16(messageBytes[0:2]))
messageBytes = messageBytes[2:]
resourceRecords[i].TTL = binary.BigEndian.Uint32(messageBytes[0:4])
messageBytes = messageBytes[4:]
resourceRecords[i].RDLength = binary.BigEndian.Uint16(messageBytes[0:2])
messageBytes = messageBytes[2:]
switch resourceRecords[i].Type {
case TypeA:
// read first 32bit/4 byte
ipv4 := net.IPAddr{
IP: messageBytes[0:4],
}
messageBytes = messageBytes[4:]
resourceRecords[i].RData = RDataA{
Address: ipv4.String(),
}
case TypeAAAA:
// read first 128bit (https://datatracker.ietf.org/doc/html/rfc3596#section-2.2)
ipv6 := net.IPAddr{
IP: messageBytes[0:16],
}
messageBytes = messageBytes[16:]
resourceRecords[i].RData = RDataAAAA{
Address: ipv6.String(),
}
case TypeCNAME:
var canonicalDomainName string
canonicalDomainName, messageBytes = parseDomainName(fullMessage, messageBytes)
resourceRecords[i].RData = RDataCNAME{
CName: canonicalDomainName,
}
case TypeMX:
rDataStr := string(messageBytes[0:resourceRecords[i].RDLength])
messageBytes = messageBytes[resourceRecords[i].RDLength:]
preference := binary.BigEndian.Uint16([]byte(rDataStr)[0:2])
var exchangeDomainName string
exchangeDomainName, messageBytes = parseDomainName(fullMessage, []byte(rDataStr[2:]))
resourceRecords[i].RData = RDataMX{
Preference: preference,
Exchange: exchangeDomainName,
}
case TypeNS:
rDataStr := string(messageBytes[0:resourceRecords[i].RDLength])
messageBytes = messageBytes[resourceRecords[i].RDLength:]
var nsDomainName string
nsDomainName, messageBytes = parseDomainName(fullMessage, []byte(rDataStr))
resourceRecords[i].RData = RDataNS{
NsdName: nsDomainName,
}
case TypeTXT:
rDataStr := string(messageBytes[0:resourceRecords[i].RDLength])
messageBytes = messageBytes[resourceRecords[i].RDLength:]
resourceRecords[i].RData = RDataTXT{
TxtData: rDataStr,
}
}
}
return resourceRecords, messageBytes, nil
}
On its own, resource records are not very useful: We want clients to be able to query DNS name servers to retrieve relevant records by specifying a domain name, record class, and type. To transmit queries like this, the RFC introduces a common format for messages.
Messages
+---------------------+
| Header |
+---------------------+
| Question | the question for the name server
+---------------------+
| Answer | RRs answering the question
+---------------------+
| Authority | RRs pointing toward an authority
+---------------------+
| Additional | RRs holding additional information
+---------------------+
Both queries and responses are transmitted using messages, as defined in Section 4.1. Messages start with a header, followed by one or more questions, resource records for answers, pointing to authoritative name servers (authority), and related information (additional) which are not strictly answers to the question.
type Message struct {
Header MessageHeader // always present
Question []QuestionEntry // question for name server
Answer []ResourceRecord // RRs answering question
Authority []ResourceRecord // RRs pointing towards authority
Additional []ResourceRecord // RRs holding additional info
}
To parse the entire message, we must start with the header and work our way through the bytes.
func parseMessage(messageBytes []byte) (Message, error) {
fullMessage := messageBytes
headerBytes := messageBytes[0:headerSizeBytes]
header, err := parseMessageHeader(headerBytes)
if err != nil {
return Message{}, fmt.Errorf("could not parse header: %w", err)
}
messageBytes = messageBytes[headerSizeBytes:]
questionEntries, messageBytes, err := parseQuestions(fullMessage, messageBytes, header.QDCount)
if err != nil {
return Message{}, fmt.Errorf("could not parse questions: %w", err)
}
answerRRs, messageBytes, err := parseResourceRecords(fullMessage, messageBytes, header.ANCount)
if err != nil {
return Message{}, fmt.Errorf("could not parse answer RRs: %w", err)
}
authorityRRs, messageBytes, err := parseResourceRecords(fullMessage, messageBytes, header.NSCount)
if err != nil {
return Message{}, fmt.Errorf("could not parse authority RRs: %w", err)
}
additionalRRs, messageBytes, err := parseResourceRecords(fullMessage, messageBytes, header.ARCount)
if err != nil {
return Message{}, fmt.Errorf("could not parse additional RRs: %w", err)
}
return Message{
Header: header,
Question: questionEntries,
Answer: answerRRs,
Authority: authorityRRs,
Additional: additionalRRs,
}, nil
}
Let’s start by parsing the message header!
Header
1 1 1 1 1 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| ID |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|QR| Opcode |AA|TC|RD|RA| Z | RCODE |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| QDCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| ANCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| NSCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| ARCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
The header section is defined in Section 4.1.1. of RFC 1035 and includes important information to identify and match the current response to a client request, determine whether the current message is a request or response (QR), specify a query kind (OPCODE), response code (RCODE), and other important flags.
type MessageHeader struct {
ID uint16 // identifier assigned by program that generated query, copied to reply
QR bool // one-bit field that specifies if message is query (0, false) or response (1, true)
OPCode OpCode // kind of query
AA bool // only in response, specifies that responding server is authority for domain name (corresponds to name matching query or first owner name in answer)
TC bool // TrunCation, specifies message was truncated due to length greater than permitted on the transmission channel
RD bool // Recursion Desired: set in query, copied to response, directs name server to pursue query recursively
RA bool // Recursion Available, set (1) or cleared (0) in response, denotes whether name server supports recursive queries
Z struct{} // reserved, must be 0
RCode RCode // response code
QDCount uint16 // number of entries in questions section
ANCount uint16 // number of RRs in answer section
NSCount uint16 // number of RRs in authority records section
ARCount uint16 // number of RRs in additional records section
}
From the specification, we already know the header to be 2 octets * 6 rows = 96 bits = 12 bytes in length. If this doesn’t match up, we can return an error.
With some bitwise operators, we can extract the relevant values from our structure, shifting bits whenever we need to access misaligned values in the octets. Using the encoding/binary package, we can translate byte sequences to numbers, which is used for parsing the ID and count values.
const headerSizeBytes = (8 * 2 * 6) / 8 // two octets * 6
func parseMessageHeader(headerBytes []byte) (MessageHeader, error) {
if len(headerBytes) != 12 {
return MessageHeader{}, fmt.Errorf("expected header to be 12 bytes")
}
id := binary.BigEndian.Uint16(headerBytes[0:2]) // first two bytes = 16 bits = 2 octets
secondRow := headerBytes[2:4] // second two bytes = 16 bits = 2 octets
qr := secondRow[0]&byte(0b1000_0000) > 0
// after consuming QR, shift entire octet to the left so that opcode takes up leading bits
secondRow[0] = secondRow[0] << 1
// retrieve four-bit opcode and shift right by remaining 4 bits in octet to "index" at 0 instead of 8
opcode := OpCode(secondRow[0] & byte(0b1111_0000) >> 4)
aa := secondRow[0]&byte(0b0000_1000) > 0
tc := secondRow[0]&byte(0b0000_0100) > 0
rd := secondRow[0]&byte(0b0000_0010) > 0
ra := secondRow[1]&byte(0b1000_0000) > 0
rcode := RCode(secondRow[1] & byte(0b0000_1111))
qdcount := binary.BigEndian.Uint16(headerBytes[4:6])
ancount := binary.BigEndian.Uint16(headerBytes[6:8])
nscount := binary.BigEndian.Uint16(headerBytes[8:10])
arcount := binary.BigEndian.Uint16(headerBytes[10:12])
return MessageHeader{
ID: id,
QR: qr,
OPCode: opcode,
AA: aa,
TC: tc,
RD: rd,
RA: ra,
Z: struct{}{},
RCode: rcode,
QDCount: qdcount,
ANCount: ancount,
NSCount: nscount,
ARCount: arcount,
}, nil
}
Once we have parsed the message header, we know how many questions, and resource records we can expect.
Question
1 1 1 1 1 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| |
/ QNAME /
/ /
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| QTYPE |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| QCLASS |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
Questions are defined in Section 4.1.2. and consist of a domain name (QNAME), a query type (QTYPE), and class (QCLASS).
func parseQuestions(fullMessage []byte, messageBytes []byte, numQuestions uint16) ([]QuestionEntry, []byte, error) {
questionEntries := make([]QuestionEntry, numQuestions)
for i := range questionEntries {
var domainName string
domainName, messageBytes = parseDomainName(fullMessage, messageBytes)
questionEntries[i].QName = domainName
questionEntries[i].QType = QType(binary.BigEndian.Uint16(messageBytes[0:2]))
messageBytes = messageBytes[2:]
questionEntries[i].QClass = QClass(binary.BigEndian.Uint16(messageBytes[0:2]))
messageBytes = messageBytes[2:]
}
return questionEntries, messageBytes, nil
}
This is all it takes to build a DNS message parser! Admittedly, it doesn’t handle cool extensions like DNSSEC, but you could go ahead and extend it easily. This goes to show that the original idea of an extensible naming system has lived up to its promises, fulfilling a critical role in powering internet infrastructure for decades to come.
I was pleasantly surprised how straightforward it was to follow the RFC and implement the parser in Go. If I find the time, I might look at other RFCs and try to implement more protocols. If you have any suggestions or questions, feel free to send a mail!