Fundamentals 18 min read

Understanding the OSI Model, TCP/IP Model, and TCP/UDP Protocols with Handshake and Teardown

This article explains the OSI seven‑layer model and the simplified four‑layer TCP/IP model, describes the TCP/IP protocol suite, details how data is encapsulated and transmitted, compares TCP and UDP, and walks through TCP’s three‑way handshake, four‑way termination, and related security considerations.

Architect's Guide
Architect's Guide
Architect's Guide
Understanding the OSI Model, TCP/IP Model, and TCP/UDP Protocols with Handshake and Teardown

OSI Seven‑Layer Model

The OSI model uses a layered architecture with seven layers: Physical, Data Link, Network, Transport, Session, Presentation, and Application.

TCP/IP Model

Because the OSI model is complex, the TCP/IP model is more commonly used in practice; it has four layers: Link, Internet, Transport, and Application. The correspondence between the two models is shown in the diagram below.

Each abstract layer builds on the services provided by the layer below and offers services to the layer above.

TCP/IP Protocol Suite

TCP/IP (Transmission Control Protocol/Internet Protocol) is the fundamental suite for the Internet, consisting of the IP protocol at the network layer and the TCP protocol at the transport layer. Although often referred to as a single protocol, it actually denotes the whole set of protocols that rely on IP for communication.

Data Encapsulation in TCP/IP Transmission

At each layer a header is added to the data, containing information required by that layer such as destination address and protocol‑specific fields. The header plus the payload together form a packet that the next lower layer treats as its data.

The packet therefore consists of two parts: the protocol‑specific header and the data passed down from the upper layer. The header’s structure is defined by the protocol specification.

Processing steps:

Application layer encodes data (presentation‑layer function) and may manage session establishment (session‑layer function).

TCP adds a TCP header to the data, establishing a reliable connection.

IP adds an IP header to the TCP segment, forming an IP packet and routing it.

The network interface (e.g., Ethernet driver) adds an Ethernet header and transmits the frame over the physical layer.

On reception, the Ethernet driver strips the Ethernet header, checks the MAC address, and forwards the payload to the appropriate module (IP, ARP, etc.).

IP verifies the destination IP, forwards the packet to the correct transport protocol (TCP, UDP).

TCP validates the checksum, reassembles the byte stream, checks sequence numbers and ports, and delivers data to the application.

The receiving application parses and displays the data.

TCP and UDP

Both TCP and UDP operate at the transport layer and rely on the IP protocol for addressing. TCP provides reliable, connection‑oriented transmission with acknowledgment, retransmission, and flow control (sliding window). UDP is connection‑less and unreliable, sending datagrams without guarantees of delivery.

Note: Many common network applications are built on TCP or UDP, but it is possible to bypass the transport layer and use IP directly (e.g., Linux LVS) or even interact with the link layer (e.g., tcpdump).

Other related protocols:

ICMP – Internet Control Message Protocol

IGMP – Internet Group Management Protocol

ARP – Address Resolution Protocol

RARP – Reverse ARP

Detailed TCP Characteristics

Connection‑oriented: a link must be established before data transfer.

Bidirectional communication once the connection is established.

Byte‑stream transmission with sequence numbers and ACKs ensures ordered, complete delivery.

Flow control via a sliding window that dynamically adjusts transmission rate.

Congestion control using slow start, congestion avoidance, fast retransmit, and fast recovery algorithms.

Sequence Number and Acknowledgment Number

Sequence number (SEQ): identifies each byte in the TCP byte stream.

Acknowledgment number (ACK): indicates the next expected byte (last received byte + 1).

Flag bits: SYN (create connection), ACK (acknowledge), FIN (terminate), RST (reset), PSH (push), URG (urgent).

TCP Three‑Way Handshake

To establish a reliable connection, TCP performs a three‑step handshake between a client (active) and a server (passive):

First handshake: Client sends SYN with initial sequence number J.

Second handshake: Server replies with SYN+ACK, acknowledges J (ACK=J+1) and chooses its own sequence number K.

Third handshake: Client sends ACK with ACK=K+1, completing the connection and entering the ESTABLISHED state.

Why Three Handshakes?

The three exchanges ensure that both sides know each other’s initial sequence numbers, guaranteeing reliable data transfer and preventing old duplicate connections from being mistakenly accepted.

SYN Flood Attack

Attackers can spoof the source IP in the first SYN packet and flood a server with SYNs, causing the server to allocate resources for half‑open connections that never complete, leading to denial of service.

TCP Four‑Way Termination

Connection teardown requires four steps:

Client sends FIN, indicating it has no more data to send (FIN_WAIT_1).

Server acknowledges with ACK and enters CLOSE_WAIT.

Server sends its own FIN when it is ready to close; it moves to LAST_ACK.

Client acknowledges the server’s FIN with ACK and enters TIME_WAIT, waiting for 2 MSL before moving to CLOSED; the server then moves directly to CLOSED.

Waiting 2 MSL ensures the full‑duplex connection is closed reliably and that delayed duplicate packets are discarded before the port can be reused.

In high‑traffic scenarios, many sockets may remain in TIME_WAIT or CLOSE_WAIT; tuning parameters such as tcp_tw_reuse and tcp_tw_recycle can help recycle TIME_WAIT sockets, while proper application code is needed to avoid lingering CLOSE_WAIT sockets.

Because establishing and tearing down connections incurs overhead, HTTP uses persistent (keep‑alive) connections to reuse an existing TCP link for multiple requests, avoiding repeated handshakes.

TCPTCP/IPnetwork protocolsUDPOSI modelconnection terminationhandshake
Architect's Guide
Written by

Architect's Guide

Dedicated to sharing programmer-architect skills—Java backend, system, microservice, and distributed architectures—to help you become a senior architect.

0 followers
Reader feedback

How this landed with the community

login Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.