I already wrote an article describing the OSI model and its 3 first layers (physical, data link and network). Now I’ll talk about the layer 4: transport. While the data link layer carries the point-to-point connections and the network layer carries the routing of packets, the transport provides end-to-end communication services for applications.
A connection between two devices on the transport layer (with TCP or UDP) is done from port to port: Each network device using the protocol TCP or UDP has different “gates” where a communication can “go in or out”. For instance, the default port to access a web server to communicate with the protocol HTTP is the port number 80, the protocol FTP uses the port number 21 and the protocol SSH uses the port number 22. (N.B. HTTP, FTP and SSH are protocol from the application layer). Applications that communicate on IP networks are bound to one (or more) port(s) to send and receive data. There are 2^16 different ports available (since the field for the port in the packet is 16 bits).
The Internet Assigned Numbers Authority has divided port numbers into three ranges.
- 0 – 1023 are used for common, well-known services.
- 1024 – 49151 are the registered ports used for IANA-registered services.
- 49152 – 65535 are dynamic ports that are not officially designated for any specific service, and can be used for any purpose. In effect, they are used as temporary ports primarily by clients when communicating with servers.
Transmission Control Protocol
TCP is a connection-oriented protocol. This means it first establishes an end-to-end communication session before any data may be send. TCP is a protocol that makes sure the data has been well delivered, in the correct order. To do so, the TCP emebed in its header a checksum verifying the data is not corrupted. There is also a sequence number to assemble the packets in the original order. Packets may use different paths to reach the recipient, or a corrupted packet need to be resend. This means the recipient might get the packet in the wrong order, the sequence number make sure when reassembling, packets are in the correct order. One other interesting feature of TCP is the windows handling. The rate of data transmission between two devices is managed by a windowing system to prevent a fast sender from transmitting more data than can be supported by the receiving.
|0||Source port||Destination port|
|64||Acknowledgment number (if ACK set)|
|128||Checksum||Urgent pointer (if URG set)|
|Options (if Data Offset > 5,padded at end with “0” bytes if necessary)
- Source port: It is the port used by client to sent the packet.
- Destination port: It is the port used by the server for the communication
- Sequence number: If the SYN flag is set, it is the initial sequence number. If it isn’t, then it is the accumulated sequence number of the first data byte of this packet for the current session.
- Acknoledgment number: If the ACK flag is set, then it is the value of the next sequence number that the receiver is expecting.
- Data offset: It specifies the size of the TCP header in 32-bit words.
- Reserved: This field is reserved for a future for a potential improvement of the protocol TCP (should be set to zero).
- Flags (NS/CWR/ECE/URG/ACK/PSH/RST/SYN/FIN): Also known as control bit, flags are notifications and gives signal the purpose of the packet. URG indicates that the Urgent pointer field is significant. ACK indicates that the Acknowledgment field is significant. All packets after the initial SYN packet sent by the client should have this flag set. RST reset the connection. SYN synchronize sequence numbers and FIN indicates no more data will be sent from sender.
- Windows Size: It specifies the number of bytes that the receiver is currently willing to receive.
- Checksum: The 16-bit checksum field is used for error-checking of the header and data.
- Urgent Pointer: If the URG flag is set, then this 16-bit field is an offset from the sequence number indicating the last urgent data byte
- Options: The length of this field is determined by the data offset field. This field is used for providing more information about the packet.
- Padding: The TCP header padding is used to ensure that the TCP header ends and data begins on a 32 bit boundary. The padding is composed of zeros.
TCP uses a three-way handshake to establish a connection between two devices (a client and a server):
- SYN: The client initiate the establishing of the communication by sending SYN to the server. It sets the segment’s sequence number to a random value A.
- SYN-ACK: In response, the server replies with a SYN-ACK. The acknowledgment number is set to one more than the received sequence number (A + 1), and the sequence number is set to another random number, B.
- ACK: Finally, the client sends an ACK back to the server. The sequence number is set to the received acknowledgement value (i.e. A + 1), and the acknowledgement number is set to one more than the received sequence number i.e. B + 1.
From now, the application can start to communicate and sending data via the session established. To close a connection, the endpoint that want to terminate the communication initiate the connection termination:
- FIN: One of the endpoint (host A) sends a FIN then wait for a ACK and a FIN from the other party (host B).
- ACK: The host B replies with an ACK.
- FIN: Then host B send a FIN.
- ACK: Finally host A ends the communication with a last ACK.
User Datagram Protocol
UDP has been design for computer applications to send messages (referred to as datagrams) to other devices without requiring prior communications to set up special transmission channels or data paths. Unlike TCP, UDP uses a simple transmission model without establishing a session with the recipient. UDP doesn’t ensure reliability, ordering, or data integrit in order to provide a faster communication. Thus UDP is used for time-sensitive applications (such as real-time systems) that don’t require 100% reliable trasmission.
UDP is a message-based connectionless protocol. It means UDP doesn’t establish a dedicated end-to-end connection but it transmitting directly information in one direction from source to destination without verifying the readiness or state of the receiver.
|0||Source port||Destination port|
- Source port: It is the port used by client to sent the packet.
- Destination port: It is the port used by the server for the communication.
- Lenght: It is the length of the UDP header and data.
- Checksum: It is optional. The checksum field is used for error-checking of the header and data. If a checksum is not used it should be set to the value zero.
Now, in order to illustrate how does TCP and UDP communicate and how packets are structured, here is a demonstration with Scapy.
UDP connection with a DNS server
As example, we will create a DNS query. When you browse a website, before establishing the TCP connection to communicate with the protocol HTTP with the server, the client need to get the IP address of the URL using DNS query:
We will try to do the same but with Scapy. Fortunately, Scapy has builded function to forge DNS query easily. Let’s create one:
>>> ls() ... DNS : DNS DNSQR : DNS Question Record DNSRR : DNS Resource Record ...
We want to create an IPv4 reverse resolution Question Record on “”. Let’s see what are the options:
>>> DNSQR().show() ###[ DNS Question Record ]### qname= '' qtype= A qclass= IN
For the IPv4 reverse resolution, the qtype is A:
>>> dnsQuery = DNSQR(qname="", qtype="A")
Now we want to create the DNS packet:
>>> dnsPacket = DNS(rd=1,qd=dnsQuery)
I will use the Google DNS (18.104.22.168) server to be sure anyone can use the same command. My IP packet then looks like this:
>>> ipPacket = IP(dst="22.214.171.124")
DNS protocol uses the port 53:
>>> port = random.randrange(49152, 65535) >>> udpPacket = UDP(sport=port, dport=53)
Let’s stack the packet and send it:
>>> packet = ipPacket/udpPacket/dnsPacket >>> dnsAnswer = sr1(packet) >>> dnsAnswer.summary() 'IP / UDP / DNS Ans "126.96.36.199" '
Here is the IP address of
Three-way handshake before a HTTP communication
As example, we will establish a session with the three-way handshake and send a HTTP GET request. When you browse a website, here is what happen:
We will try to do the same but with Scapy. For the tree-way handshake, you first need to create an IP packet with an destination IP address:
>>> ipPacket = IP(dst="")
Now we create the TCP packet with a source port between 49152 and 65535 (see the introduction for the reason) and the destination port 80 (the HTTP port):
>>> port = random.randrange(49152, 65535) >>> tcpPacket = TCP(sport=port, dport=80, flags="S")
random() is a python module that generates pseudo-random numbers. randrage() set the range of the pseudo-random number.
Once both pakets created, we stack them to create the SYN TCP packet:
>>> syn = ipPacket/tcpPacket
Then we send the SYN packet and store the SYN/ACK answer:
>>> synack = sr1(syn) >>>() ###[ IP ]### version= 4L ihl= 5L tos= 0x0 len= 44 id= 0 flags= DF frag= 0L ttl= 52 proto= tcp chksum= 0xb760 src= 188.8.131.52 dst= 192.168.0.7 options= '' ###[ TCP ]### sport= http dport= 52708 seq= 3872114628 ack= 1 dataofs= 6L reserved= 0L flags= SA window= 14600 chksum= 0x52dc urgptr= 0 options= [('MSS', 1460)] ###[ Padding ]### load= '\x00\x00'
No the last part of the handshake is to send an ACK with the sequence number set to the value of the acknoledgement number of the SYN/ACK and the acknoledgement number set to the sequence number + 1 of the SYN/ACK:
>>> ack = ipPacket/TCP(sport=port, dport=80, flags="A") >>> ack[TCP].seq=synack[TCP].ack >>> ack[TCP].ack=synack[TCP].seq+1 >>> send(ack)
The session is now established. Let’s send a HTTP GET request and store the answers:
>>> httpRequest = 'GET / HTTP/1.1\r\nHost:' >>> tcpData = TCP(sport=port, dport=80, flags="A") >>> [TCP].ack >>> [TCP].seq+1 >>> httpPacket = ipPacket/tcpData/httpRequest >>> answers,unanswered = sr(httpPacket)
Here is a script that will run automatically all those commands:
#!/usr/bin/python # Import scapy fromimport * # Froge the IP packet ipPacket = IP(dst=" ") # Generate the pseudo-random source port port = random.randrange(49152, 65535) # Forge the TCP packet for the SYN tcpPacket = TCP(sport=port, dport=80, flags="S") # Create the packet to send syn = ipPacket/tcpPacket # Send the packet and store the answer synack = sr1(syn) # Forge the packet to send based on the answer ack = ipPacket/TCP(sport=port, dport=80, flags="A") ack[TCP].seq=synack[TCP].ack ack[TCP].ack=synack[TCP].seq+1 send(ack) # Forget the HTTP packet to send httpRequest = 'GET / HTTP/1.1\r\nHost: ' tcpData = TCP(sport=port, dport=80, flags="A") [TCP].ack [TCP].seq+1 httpPacket = ipPacket/tcpData/httpRequest answers,unanswered = sr(httpPacket) ()
sudo python the-name-of-the-file.py to run the script.
You will notice that the script actually doesn’t work. I’m not sure about it (and please leave a message if I’m wrong), but it seems Scapy uses an other TCP stack than the kernel. This means whenever Scapy sends the SYN packet, the kernel is not aware of it, so when the server answer with a SYN/ACK, the kernel also get the packet and doesn’t understand why the server sent a SYN/ACK. Therefore, it send a RST TCP packet to reset the connection.
In order to avoid this problem, I created a rule in my firewall (on my VM Ubuntu 10.10) to drop all RST packets going out.
sudo iptables -A OUTPUT -i eth0 -s 192.168.0.7 -p tcp --tcp-flags RST RST -j DROP
eth0 being the interface used for sending the packet with scapy, 192.168.0.7 being the IP of the my VM that send the packet.
Now if you try script, you will see the TCP connection is established and that HTTP GET request (nearly) works. The server is waiting for an ACK once the window reached, since it doesn’t get any ACK, it keeps resend the packets.
Well the main goal was to describe what is the layer transport and how does the two main protocols of this layer (i.e. TCP and UDP) work. The script is not finish yet, but you got an idea how scapy works and how to establish a TCP connection. I think I will create another post to improve the script an also use pfctl to avoid the RST problem on OS X.
I hope you enjoyed, feel free to leave a feedback