System Design Series — Network Protocols

Refresher on Key Network Protocols

HTTP/HTTPS

HTTP stands for HyperText Transfer Protocol. It is an application layer protocol designed to encode and transport information between networked devices and runs on top of other layers of the network protocol stack. HTTPS is the secure version of HTTP. HTTPS is encrypted in order to increase the security of data transfer.

Example: When we enter a URL in your browser, HTTP/HTTPS protocols are used to fetch and display the web page.

TCP

TCP stands for Transmission Control Protocol. It is a transport layer protocol that enables application programs and computing devices to exchange messages over a network. It is designed to send packets across the internet and ensure the successful delivery of data and messages over networks. TCP is a connection-oriented & reliable protocol.

Example: TCP is used in applications where reliability is crucial, such as web browsing and email.

UDP

User Datagram Protocol (UDP) is a Transport Layer communication protocol, used across the Internet for time-sensitive transmissions such as video playback or DNS lookups. It speeds up communications by not formally establishing a connection before data is transferred.

Example: UDP is often used in live streaming and online gaming where speed is more critical than reliability.

WebSocket

WebSocket is a communication protocol that allows a web server and a client, like a web browser, to exchange data in real-time. It’s different from the traditional HTTP method of loading web pages, which requires a separate request and response for each piece of data. The WebSocket API makes it possible to open a two-way interactive communication session between the user’s browser and a server.

Example: WebSockets are commonly used in applications requiring real-time updates, like chat apps and online gaming.

WebRTC

WebRTC (Web Real-Time Communication) is an open-source technology created by Google that enables browser-to-browser real-time communication and data exchange, primarily focused on audio and video traffic. These are set of javascript APIs.

• User 1 makes an offer using Session Description Protocol
• User 2 accepts the offer and writes a SDP response object to signaling server
• Clients use ICE to publish their internet routable IP address using the Session Traversal utilities for NAT (STUN server).

WebRTC uses UDP under the hood.

Example: WebRTC is used in video chat applications and real-time data sharing between browsers.

IP Address protocol (v4/v6)

Internet Protocol versions 4 and 6 are used for routing and addressing packets of data so they can travel across networks and arrive at the correct destination.

Example: IPv4 uses 32-bit addresses, while IPv6 uses 128-bit addresses, allowing for a much larger number of unique addresses.

FTP

File transfer protocol (FTP) is a standard communication protocol used to transfer computer files from one location to another over a TCP-based network, such as the internet.

Example: FTP is used to upload files to a website’s server.

SSH

The Secure Shell (SSH) protocol is a method for securely sending commands to a computer over an unsecured network. SSH uses cryptography to authenticate and encrypt connections between devices. SSH provides password or public-key based authentication and encrypts connections between two network endpoints

Example: SSH is used by administrators to log into remote servers securely.

RPC

Definition: Remote Procedure Call (RPC) allows a program to execute a procedure on a remote server as if it were local. It is a network programming model or interprocess communication technique that is used for point-to-point communications between software applications. Client and server applications communicate during this process.

Example: There are different RPC frameworks , and gRPC is one example of an RPC framework that is widely used in modern software development.

DNS

The Domain Name System (DNS) is the phonebook of the Internet. Humans access information online through domain names, such as nytimes.com or espn.com. Web browsers interact through Internet Protocol (IP) addresses. DNS translates human-readable domain names (for example, www.amazon.com) to machine-readable IP addresses (for example, 192.0. 2.44).

Example: When we type a URL in our browser, DNS resolves the domain name to the IP address of the web server.

DHCP

Dynamic Host Configuration Protocol (DHCP) is a network protocol used to automate the process of configuring devices on IP networks, thus allowing them to use network services such as DNS, NTP, and any communication protocol based on UDP or TCP. DHCP helps our hosts/systems obtain a unique identity over a network. We call this identity as IP address. DHCP provides much more information than those like subnet mask, default gateway, DNS addresses, etc. Below is a high-level diagram of how a client interacts with the DHCP server.

Example: DHCP automatically assigns a new IP address to your computer when you connect to a Wi-Fi network.

MQTT

MQTT stands for Message Queuing Telemetry Transport. It is a lightweight messaging protocol for use in cases where clients need a small code footprint and are connected to unreliable networks or networks with limited bandwidth resources.

Example: MQTT is commonly used in IoT applications for sending messages between devices.

AMQP

The Advanced Message Queuing Protocol (AMQP) is an open standard for passing business messages between applications or organizations. It connects systems, feeds business processes with the information they need and reliably transmits onward the instructions that achieve their goals.

Example: AMQP is used in financial services for handling high-value, high-speed transactions.

SMTP

SMTP stands for Simple Mail Transfer Protocol. It is a communication protocol used for sending and receiving email messages over the Internet. Mail servers and other message transfer agents (MTAs) use SMTP to send, receive and relay mail messages. It operates at the application layer of the TCP/IP protocol stack.

Example: When you send an email, SMTP is used to transmit your message to the recipient’s mail server.

IMAP

The Internet Message Access Protocol (IMAP) is a protocol for receiving email. Protocols standardize technical processes so computers and servers can connect with each other regardless of whether or not they use the same hardware or software.

Example: IMAP allows you to access your email from multiple devices by keeping emails on the server.

POP3

POP3 is a standard email protocol that allows users to retrieve email messages from a mail server onto their local device, such as a computer or mobile phone. When an email is sent to a user’s email address, it is first stored on the email server. The POP3 protocol allows the user’s email client to access the email server, download the email messages, and delete them from the server.

Example: Unlike IMAP, POP3 downloads email to a single device and deletes it from the server.

OSPF

Open Shortest Path First (OSPF) is a link-state routing protocol that is used to find the best path between the source and the destination router using its own Shortest Path First). OSPF was developed by Internet Engineering Task Force (IETF) as one of the Interior Gateway Protocol (IGP), i.e, the protocol which aims at moving the packet within a large autonomous system or routing domain.

Example: OSPF is widely used in large enterprise networks for efficient routing.

Also read: BGP, Distance vector routing, Link State routing, RIP

SNMP

SNMP stands for “Simple Network Management Protocol.” It’s an application layer protocol included in the Internet protocol suite, a set of the most commonly used communication protocols online. The SNMP protocol is embedded in multiple local devices such as routers, switches, servers, firewalls, and wireless access points accessible using their IP address. SNMP provides a common mechanism for network devices to relay management information within single and multi-vendor LAN or WAN environments.

Example: SNMP is used for network management, monitoring network performance, and detecting network faults.

ICMP

The Internet Control Message Protocol (ICMP) is a set of communication rules that devices use to communicate data transmission errors in a network. In a message exchange between the sender and receiver, certain unexpected errors can occur. For example, messages can be too long, or data packets can arrive out of order so the receiver can’t assemble them. In such cases, the receiver uses ICMP to inform the sender with an error message and requests the message be resent.

Example: ICMP is used in the ping command to test the reachability of a host.

ARP

Address Resolution Protocol (ARP) is a protocol or procedure that connects an ever-changing Internet Protocol (IP) address to a fixed physical machine address, also known as a media access control (MAC) address, in a local-area network (LAN).

Example: ARP is used when a device needs to find the MAC address of another device on the same local network.