Internet Protocols
3 March 2024
The Internet is a network of end systems transmitting data or packets with each other. An end system connects to the Internet at the network edge, through access networks provided by an ISP. Once connected, the end system can start sending packets into the network core. Packets in the network core travel across a path from source to destination, through a series of packet switches. Much of the network core is just a collection of packet switches that are partitioned and managed by the different ISPs.
In addition to these physical components, the Internet is also powered by a collection of protocols. Internet protocols are organised into layers; upper layer protocols are implemented using lower layer protocols. For example, HTTP uses TCP as its underlying transport protocol, which in turn uses IP, and so on. Common Internet protocols are HTTP, DNS, TCP, UDP, and IP; all follow standard documents called RFCs drafted by IETF.
The top most layer in the Internet protocol stack is the application layer. Applications, or network applications, are programs that run on different end systems and communicate with each other over the Internet. A running instance of a program, also called a process, passes messages into the Internet through sockets, ie. the interface between the application layer and the transport layer immediately below. When a client process passes a message into a socket, it must also pass the IP address of the destination end system, and the port number of the server process running on that system.
HTTP is one example of an application layer protocol. HTTP defines the format of messages that are exchanged by Web applications running on the Internet. There are two kinds of HTTP processes: HTTP clients and HTTP servers. The HTTP server exposes a set of services through an API. The HTTP client consumes this API by sending an HTTP request containing a payload, a set of headers, and an (HTTP_METHOD, URL) pair. Once an HTTP request is received, the server processes it, and then sends back an HTTP response containing a payload, a set of headers, and a status code or message. HTTP also provide other functionalities: a persistent flag to indicate whether to establish a persistent connection, cookies to keep track of clients, and conditional GETs to implement web caching. HTTP uses TCP as its underlying transport layer protocol.
DNS is another example of an application layer protocol. DNS is a directory service used for translating hostnames to IP addresses. Thus, a client process, running on a source end system, trying to communicate with a server process, running on a destination end system, can use DNS to obtain the IP address of the destination end system. DNS also provides other services such as host aliasing, mail aliasing, and load distribution (rotating IP addresses). DNS is implemented as a hierarchy of DNS servers: root DNS servers, top-level domain DNS servers, authoritative DNS servers (maintained by the application server administrators), and local DNS servers (maintained by the ISPs). An application program accesses DNS by sending a DNS query to a local DNS server, which acts as local proxies for the servers in the actual DNS hierarchy. Non-local DNS servers store resource records containing mapping information. There are four types of resource records: Type A (hostname-IP), type NS (domain-hostname), type CNAME (alias-canonical), and type MX (alias-canonical mail). DNS uses UDP as its underlying transport layer protocol.
The next layer in the protocol stack is the transport layer. When a network application passes a message into a socket, the transport layer breaks them into smaller chunks, and encapsulates each with header information to create segments. The two most common transport layer protocols are UDP and TCP.
UDP is a lightweight protocol. It is fast, but not always reliable. UDP encapsulates application data with a minimal set of headers that includes the source port number, the destination port number, the length of the UDP segment, and a checksum value for error detection. No connection is established between the source and destination, and no connection state is maintained.
TCP is a transport layer protocol that offers more guarantees. Most notably, it provides reliable data transfer. TCP segments are guaranteed to arrive in order and without corruption. TCP is connection oriented, and it leverages the mechanisms of reliable data transfer, such as checksums (for error detection), acknowledgements (to handle retransmissions in case of missing acknowledgements), sequence numbering (to handle packet duplicates due to retransmissions), and timeouts (to handle packet losses). TCP also implements pipelining (along with GBN to recover from pipeline errors) to increase utilisation and throughput. Furthermore, TCP implements congestion control to throttle the sender in case of a network congestion (high costs due to increased queueing delays, higher packet losses, and increased retransmissions). TCP reacts to network congestions by adjusting the window size. After a connection has been established, TCP sends segments into the network layer using a window that increases exponentially. When a segment is lost for the first time, TCP enters congestion avoidance mode and sets the window size to half the value of that during the loss event. TCP also provides fast recovery: when a segment is lost and multiple duplicate ACKs are received by the sender, TCP increases the window size gradually under the ACK for the missing segment is received, and then returns to congestion avoidance mode again.
Both UDP and TCP are built on top of the network layer, which is responsible for providing host-to-host delivery as opposed to process-to-process delivery of the upper application and transport layers. The network layer achieves this through two mechanisms: forwarding and routing. Forwarding is the process of moving a packet from an inbound link to the correct outbound link, and is typically implemented in hardware. Routing is the process of determining the path that packets should take from the source to the destination. The two are closely related by the forwarding table, which is used by the router to determine the outbound link for a packet. Forwarding tables are set by routing protocols, such as OSPF and BGP.
One key protocol in the network layer is the IP protocol. IP defines the format of datagrams that are exchanged between routers (ie. the source and destination IP addresses in the headers), and the rules for how routers should forward and route those datagrams. An IP address is associated with an interface. Each address is 32 bits long, written in dotted decimal notation. CIDR is used to partition the 32-bit IP address space into smaller blocks. Each block represents a network or subnet. For example, in a.b.c.d/x, the x most significant bits are used to represent the network, and the remaining bits are used to represent the hosts in the network. Organisations obtain IP address blocks from various parties such as IANA and various regional Internet registries, all collectively managed by ICANN.
Once an organisation has been allocated an IP address block, network administrators in the organisation can begin to set up routers and manually assign IP addresses to the interfaces of the routers. Host devices that connect to the organisations' networks, on the other hand, are assigned IP addresses dynamically using DHCP. DHCP is a client-server protocol. Each subnet has a DHCP server (or a DHCP relay agent), which is responsible for assigning IP addresses (permanent or temporary) to new hosts in the subnet. IP allocation happens in four steps: (1) the client broadcasts a HDCP discover message to the subnet; (2) the server responds with a DHCP offer message, containing an IP address and other network configuration parameters; (3) the client sends a DHCP request to the server; (4) the DHCP server sends a DHCP ACK to the client, confirming the allocation.
IP also reserves a set of addresses for special purposes. One such set is reserved for use in local private networks. These private addresses are not publicly routable. They can be accessed from the outside through a NAT router. NAT routers have their own IP address, and they maintain a table of mappings between the private IP addresses and the NAT router's IP address. When a client in the private network sends an IP datagram to the outside, the NAT router replaces the source IP address of the datagram with its own IP address, generates a new port number and stores it in the NAT translation table, and then forwards the datagram to the outside. When the outside host responds, the IP datagram is sent to the NAT router, which then looks up the NAT translation table to find the private IP address and port number of the client, and then forwards the datagram to the client.
Other protocols in the network layer are OSPF and BGP. Unlike IP, which defines the forwarding and routing rules on a single router, OSPF and BGP are used to set up the forwarding tables across multiple routers. These protocols use link-state and path-vector algorithms to determine the shortest path between the source and destination.