As a programmer, it is impossible not to deal with the Internet. Now our mobile phone, computer, it is no exaggeration to say, leaving the network is a piece of 'scrap iron', their role will be greatly reduced.. The role of this article, mainly For those who are not non-network professional development, take the premise of 'the shortest time, know the most knowledge of the network'.
Let’s first understand the meaning of various professional terms that we know, but don’t quite understand.
Internet
Internet
The Internet is the largest network in the world today, and it is a "network of networks." That is, the Internet is a giant network interconnected by all networks.
The composition of the Internet:
Edge part: host
Core: A large number of networks and routers that connect to these networks (this router is not our home router)
Ethernet
Ethernet is the most commonly used LAN communication protocol. Ethernet frames are transmitted on the Ethernet. Since Ethernet only allows one computer to send data at the same time, there must be a detection mechanism, which is the CSMA/CD protocol:
Multi-point access: Multiple computers are connected to one bus in a multipoint access mode
Carrier monitoring: Each station must constantly detect the channel regardless of whether it is transmitting or not
Collision detection: Listening while sending
OSI
Open System Interconnection Basic Reference Model, any two systems can communicate as long as they comply with this OSI standard. OSI is a seven-layer protocol architecture, and TCP/IP is a four-layer protocol architecture, so we take a compromise approach. When learning the principle of computer network, the architecture of the five-layer protocol is often used: physical layer, data link layer, network layer, transport layer and application layer.
Protocol architecture
Physical layer
There are only 0 and 1 in the world of computers. As you read the text of this article, storing in a computer is also a bunch of combinations of 0 and 1. But these numbers cannot be transmitted in real physical media, but need to be Converted to optical or electrical signals, so this layer is responsible for converting these bit streams (0101) and photoelectric signals.
If there is no physical layer, then there is no Internet, there is no sharing of data, because data cannot flow in the network.
data link layer
The data is no longer transmitted in the form of a bit stream at this layer, but is divided into frames one by one and then transmitted.
MAC address
Also known as the computer's hardware address, the 48-bit address is fixed on the adapter (network card) ROM. The MAC address can be used to uniquely distinguish a computer because it is unique in the world.
Packet switching
Since the data is to be divided into frames one at a time, since different links specify different maximum frame lengths, ie MTUs (maximum transmission units), frames that exceed this MTU must be partitioned. For example, A truck can transport 5 tons of cargo at a time, and a road has a limited load of 2 tons, then you have to transport it in 3 times.
Bridge
The bridge works at the data link layer, and forwards and filters the received frames according to the destination address of the MAC frame.
Ethernet switch
In fact, it is a multi-interface bridge. Each interface of an Ethernet switch is directly connected to a single host or another hub, which can easily implement VLAN (Virtual Local Area Network).
Ethernet MAC frame
The format of the MAC frame is:
MAC frame format
Destination address: Receiver's 48-bit MAC address
Source address: The sender's 48-bit MAC address
Type field: What protocol is used on the upper layer of the flag, 0×0800 is the IP datagram.
Network layer
If only the data link layer has no network layer, data can only be transmitted on the same link and cannot be transmitted across links. With the network layer, data can be transmitted across different data links.
IP address
The IP address, also known as the software address, is stored in the computer's memory, with an IPv4 address of 32 bits and an IPv6 address of 128 bits.
IP address and MAC address
IP address is used above the network layer, and MAC address is used below the data link layer.
The IP address is a logical address, and the MAC address is a physical address.
The source and destination addresses of the header in the IP packet will not change during transmission. The source and destination addresses of the header in the MAC frame will change once per router.
IP address classification
IP address = {<network number>, <host number>}
Class A address: 0.0.0.0 ~ 127.0.0.0 Class B address: 128.0.0.0 ~ 191.255.0.0 Class C address: 192.0.0.0 ~ 223.255.255.0
IP address after subnetting
IP address = {<network number>, <subnet number>, <host number>}
For example, a unit has a Class B IP address, 145.13.0.0, but any datagram with a destination address of 145.13.xx will be sent to the router on this network. R. Internal subnetting becomes: 145.13.3.0, 145.13.7.0 , 145.13.21.0. But the external still appears as a network, that is, 145.13.0.0. After the router R receives the message, it sends it to the corresponding subnet according to the destination address.
Subnet mask
Generally consists of a string of 1 and a string of 0, regardless of whether the network has subnetted, the subnet mask and IP address are bitwise AND operation to get the network address.
All networks must use a subnet mask and must have a subnet mask in the routing table. If a network is not subnetted, the subnet mask of that network is the default subnet mask.
The default subnet mask for class A addresses is 255.0.0.0B. The default subnet mask for class B addresses is 255.255.0.0. The default subnet mask for class B addresses is 255.255.255.0.
Although subnetting adds flexibility, it reduces the total number of hosts that can be connected to the network.
The IP address that constitutes the supernet
IP address = {<network prefix>, <host number>}
Use network prefix, no classification inter-domain routing CIDR
For example, 128.14.35.7/20, meaning that the first 20 bits are the network prefix and the last 12 bits are the host number. In addition, CIDR forms a "CIDR address block" with consecutive IP addresses with the same network prefix.
Address Mask: CIDR uses a 32-bit address mask, similar to a subnet mask.
IP datagram
At the network layer, data is transmitted in the form of IP datagrams (IP packets).
IP datagram format
The first 20 bytes of the header are fixed length, which is mandatory for all IP datagrams. The last 4 bytes are optional fields, and their length is variable.
IP datagram header fixed field analysis:
Version number: IP version, IPv4 or IPv6
Header length: The length of the header is recorded, up to 1111, which is 15 32-bit words long, ie 60 bytes. When the header length is not an integer multiple of 4 bytes, it needs to be filled with the last padding field.
Service Type: Generally useless
Total length: refers to the length of the sum of the header and the data. The maximum is 216-1 = 65535 bytes. However, since the data link layer specifies the maximum length MTU of each frame, the Ethernet specifies the MTU to be 1500 bytes. Therefore, datagrams that are out of range must be sliced.
Identification: Each time an IP datagram is generated, the counter is +1, and this value is assigned to the identification field. In the datagram that needs to be fragmented later, the same description is the same datagram.
Flag: 3 digits, the lowest digit is MF (More Fragment). MF = 1 indicates that there are fragments; MF = 0 indicates that this is the last fragment. The middle one is recorded as DF (Don't Fragment), This means that it cannot be sliced. Only when DF = 0, fragmentation is allowed.
Segment displacement: Also known as slice displacement, where the slice starts from the start of the user data field. The slice displacement is in 8-byte offset units. Therefore, the length of each slice must be an integer of 8 bytes. Times.
Time to live: TTL (Time To Live). The maximum number of times a datagram can pass through the router on the Internet is 255 times. Each time a router passes TTL – 1, when it is 0, the message is discarded.
Protocol: Record which protocol is used to carry the data.
The first checksum: Only the header of the datagram is checked, the data part is not checked. If it is not 0, the message is discarded.
Source address and destination address: do not explain
IP layer forwarding packet flow
Each router maintains a routing table inside. The routing table contains the following contents (destination network address, next hop address).
When using subnet packet forwarding, the routing table must contain the following three items: destination network address, subnet mask and next hop address.
Specific host route: Specify a route to a specific destination address
Default route: I don't know which router the packet should be sent to the default route. It is very suitable to use the default route when there is only a few external connections in a network.
Router packet forwarding algorithm
Get the destination IP address D from the datagram, and get the destination network address N.
If N is a network address directly connected to this router, it will be delivered directly (no need to be forwarded to other routers for forwarding, directly to the destination host for delivery), otherwise -> (3)
If there is a specific host route with the destination address D in the routing table, the datagram is transmitted to the router, otherwise -> (4)
If there is a route to the network N in the routing table, the datagram is transmitted to the router, otherwise -> (5)
If there is a default route in the routing table, it is given to the router, otherwise -> (6)
Report forwarding packet error
Virtual private network VPN
All routers on the Internet will not forward datagrams whose destination address is a private address. There are three kinds of private addresses (virtual IP addresses).
10.0.0.0 ~ 10.255.255.255
172.16.0.0 ~ 172.31.255.255
192.168.0.0 ~ 192.168.255.255
Suppose now that company A has one department in Guangzhou and another in Shanghai, and they have their own private network in the local area. So how do you connect these two private networks?
The communication line for renting telecommunications is dedicated to the agency, but it is too expensive.
Using the public Internet as a communication carrier, this is the virtual private network VPN.
Network address translation NAT
The hosts inside multiple private networks share the IP address of a NAT router. When the host sends and receives IP datagrams, it must first perform network address translation through the NAT router.
How NAT routers work
Not only that, but NAT can also use the port number to transform into a network address and port to convert NAPT.
ARP protocol
ARP is to solve the mapping problem between the IP address and MAC address of the host or router on the same LAN, ie IP address -> ARP -> MAC address
Each host has an ARP cache with a mapping table from the IP address to the MAC address of each host and router on the LAN. The following is how ARP works:
How ARP works.jpg
So what if you use ARP across networks?
Broadcast on this network
If the host is not found, then to the router
The router helps to forward (broadcast on another network)
If found, the ARP request is completed, and if it is not found, it returns (2)
Transport layer
This layer is the most important, because the data link layer, the network layer two layers of data transmission are unreliable, do their best to deliver. What does it mean? They are not responsible for submitting to you is correct Data. However, this layer of TCP protocol will provide reliable transmission.
The main focus of this layer is two protocols: UDP and TCP.
User Datagram Protocol UDP
Main features of UDP:
no connection
Do your best to deliver
For message: The message handed over by the application layer is directly added to the IP layer by adding the UDP header, and is not merged or split.
No congestion control
Support one-to-one, one-to-many, many-to-one and many-to-many interactive communication
Header overhead is small, only 8 bytes
UDP header
UDP header format
Source port: Source port number. Select when you need the other party's reply, if not, then all 0
Destination port: Destination port number. This must be used when delivering the message at the destination.
Length: The length of the UDP datagram, the minimum is 8 (only the header)
Checksum: Unlike IP datagrams, which only test the header, UDP needs to check the header and data part together.
Transmission Control Protocol TCP
Main features of TCP:
Connection-oriented transport layer protocol
Each TCP connection can only have 2 endpoints, and TCP is peer-to-peer.
Provide reliable delivery
Full duplex communication
Byte-oriented stream
TCP workflow
TCP byte stream
TCP connection
The endpoint of the TCP connection is called a socket.
Socket = (IP address: port number)
Each TCP connection is uniquely determined by two endpoints at both ends of the communication. That is: TCP connection::= {socket1, socket2} = {(IP1 : port1), (IP2 : port2)}
The header of the TCP segment
The header of the TCP segment
Source port and destination port: function with UDP port
Serial number: The serial number of the first byte of the data in this segment
Confirmation number: If the number of the first data byte of the next segment of the other party is expected to be received, if the confirmation number = N, it means: All data up to the sequence number N-1 has been received normally.
Data offset: the length of the TCP segment
Reserved: Used later, currently 0
Emergency URG: If URG = 1, the emergency pointer field is valid, telling the system that this is urgent data should be transmitted as soon as possible. For example, the transmission is suddenly interrupted.
Acknowledge ACK: ACK = 1, the acknowledgment number is valid, ACK = 0, the acknowledgment number is invalid. TCP stipulates that all transmitted segments must be set to ACK after the connection is established.
Push PSH: If PSH = 1, the receiver will not be able to wait until the entire cache is full after receiving the segment.
Reset RST: When RST = 1, the TCP connection has a serious error, the connection must be released and then reconnected.
Synchronous SYN: Used to synchronize the sequence number when the connection is established. When SYN = 1, ACK = 0, it indicates that this is a connection request segment. If the other party agrees to establish a connection, set SYN = 1 in the response segment. ACK = 1
Terminate FIN: When FIN = 1, it indicates that the sender data of this segment has been sent and asks to release the connection.
Window: Tell the other party: From the confirmation number in the header of this segment, the amount of data that the receiver currently allows the other party to send. This is the basis for the receiver to let the sender set its send window.
Checksum: Same as UDP, check header and data section
Urgent pointer: Valid when URG = 1, indicating the end of the urgent data at the end of the segment
Options: Up to 40 bytes, no 0 is the maximum segment length MSS (Maximum Segment Size): The maximum length of the data field in each TCP segment, if not filled, the default is 536 bytes.
window
A very important concept in TCP, that is the window (send window and receive window)
window
Since the stop-and-wait protocol is very inefficient, the concept of a window is derived. The above picture shows the transmission window maintained by the sender. The five packets located in the transmission window can be sent continuously without waiting for the confirmation of the other party. Confirm, just move the send window forward one block position. This greatly improves the channel utilization!
The receiver does not have to send an acknowledgment message for each packet, but instead uses a cumulative acknowledgment. That is, an acknowledgment message is sent for the last packet that arrives in order.
Timeout retransmission
If the sender does not receive an ACK acknowledgment message after waiting for a period of time, a timeout retransmission will be initiated. This waiting time is the retransmission timeout (RTO, Retransmission TimeOut).
However, the value of RTO is not fixed. This time is always slightly larger than the Round Trip Time (RTT). It is assumed that the message transmission takes 5 seconds in the past, and it takes 5 seconds to send the confirmation message after the other party receives it. RTT is 10 seconds, then the RTO is slightly larger than 10 seconds. Then after the RTO has not received the confirmation message, the message is lost, it will be re-transmitted.
flow control
Use the sliding window and the sending timing of the segment to control the flow.
Congestion control
The sender maintains a congestion window cwnd, send window = congestion window.
Slow start: cwnd = 1, then double the congestion avoidance after each transmission round: Let cwnd increase slowly, +1 slow start threshold ssthresh after every transmission round:
When cwnd < ssthresh, use the slow start algorithm when cwnd > ssthresh, use the congestion avoidance algorithm when cwnd = ssthresh, free
Congestion control
As long as it is judged that the network is congested, set ssthresh to half of the current congestion window (not less than 2), and set cwnd to 1, and re-execute the slow start algorithm.
In addition to the slow start and congestion avoidance algorithms, there is a set of fast retransmission and fast recovery algorithms:
Fast retransmission: The receiver sends an acknowledgment in time, and the sender only needs to receive three duplicate acknowledgments, and immediately retransmits the fast recovery: When the sender receives three repeated acknowledgments, ssthresh is halved and cwnd is set to ssthresh.
TCP three-way handshake
TCP three-way handshake to establish a connection and four wave disconnects is the knowledge point of the interview.
TCP three-way handshake
Q: Why do you have to shake hands three times, not twice?
A : Imagine, A sends the request connection for the first time, but it is stuck in a node on the network, A times to retransmit, then this time everything is normal, A and B are happy to transfer data. Wait until the connection is released, that The lost connection request suddenly arrives at B. If it is a two-way handshake, B sends a confirmation, even if they establish a connection. In fact, A does not care about this confirmation, because I have no data to pass. B is stupid to think that there is data to come, waiting for it. The result is a waste of resources.
The more grounded explanation is: A call B
First handshake: Hello, I am A, can you hear me talking? The second handshake: I heard, I am B, can you hear me talking? The third handshake: I heard, we can start chatting The three-way handshake is actually to test whether the sending and receiving capabilities of both parties are normal. What do you say?
TCP waved four times
TCP waved four times
Q: Why do you have to wave four times instead of twice, three times?
A: First of all, due to TCP full-duplex communication, both parties can act as data senders. A wants to close the connection, you must wait for the data to be sent before sending FIN to B. (At this time, A is in a semi-closed state) B sends an acknowledgment ACK, and B sends it if it wants to send data (for example, some pre-release processing). After B sends the data, it sends FIN to A. (B is half-closed at this time) A sends an ACK, enters the TIME-WAIT state, and after receiving the message from B after 2MSL time, it is determined that B has received the ACK. (At this time, A and B are considered to be completely closed)
PS : Carefully analyze the above steps to know why you can't wave less than four times.
Q: Why wait for 2MSL (Maximum Segment Lifetime) time, from TIME_WAIT to CLOSED?
A : The last ACK reply is sent at the Client, but the ACK may be lost. If the Server does not receive an ACK, it will repeatedly send FIN fragments repeatedly. So the Client cannot close immediately, it must confirm that the Server received the ACK. The Client will enter the TIME_WAIT state after sending out an ACK. The client will set a timer and wait for 2MSL. If FIN is received again within this time, the client will resend the ACK and wait for 2MSL again. MSL refers to the maximum lifetime of a segment in the network, and 2MSL is the maximum time required for a transmission and a reply. If the client does not receive the FIN again until 2MSL, then the client concludes that the ACK has been successfully received, and ends the TCP connection.
More grounded explanation:
The first wave: A tells B, I have no data, I am ready to close the connection, you have to send data? The second wave: B sends the last data for the third time: B tells A, I have to close the connection. The fourth wave: A tells B that you can close, and I am closed here.
Application layer
The most famous application layer protocol is HTTP, FTP, and an important DNS.
Domain Name System (DNS)
DNS can resolve a domain name (for example, ) to an IP address.
Domain name server classification
Root Domain Name Server: The highest level of domain name server
Top level domain server: as its name
Permissions domain name server: Responsible for a zone's server
Local domain name server: The host sends a DNS query request to it.
DNS query
DNS query
The host's query to the local domain name server is generally a recursive query.
The local domain name server queries the root domain name server usually by iterative query.
Recursive query: B asks how Guangzhou Guangzhou goes, A doesn't know, A asks C, C doesn't know and asks D... until I know it, I will tell you one by one until A tells B. Iterative query: B asks A Guangzhou how to Go, A don't know, A tells you to ask C, then B asks C, C doesn't know, C tells you to ask D, then B asks D... until B knows
DNS query example: The host with the domain name x.tom.com wants to know the IP address of y.jerry.com
The host x.tom.com first recursively queries the local domain name server dns.tom.com
The local domain name server uses an iterative query. It first asks for a root name server.
The root domain name server tells it, you go to the top-level domain server dns.com
Local domain name server asks top-level domain server dns.com
The top-level domain server tells it, you go to the permission domain name server dns.jerry.com
Local domain name server asks permission domain name server dns.jerry.com
The privilege name server dns.jerry.com tells the IP address of the host it is queried
The local domain name server tells the host x.tom.com
PS : This query uses UDP, and in order to improve DNS query efficiency, each domain name server uses a cache.
URL
URL format: <protocol>://<host>:<port>/<path>, port and path can sometimes be omitted.
URL using HTTP protocol: http://<host>:<port>/<path>, HTTP default port number is 80
HTTP protocol
HTTP is transaction-oriented, that is, the data it transmits is a whole, either all received, or not all received.
The working process of the World Wide Web
Each HTTP request requires a TCP connection and a TCP connection to be established.
HTTP is connectionless, stateless. Every request is a new request.
HTTP/1.0 Disadvantages: No connection, re-establish TCP connection every request, so every HTTP request takes 2 times RTT time (one TCP request, one HTTP request)
HTTP/1.1: Use persistent connection, that is, keep the TCP connection for a while.
Two working modes of HTTP/1.1 continuous work: Non-pipelined mode and pipelined mode Non-pipelined mode: Receive a request response and send another request, which is inefficient and wastes resource pipeline mode: It can send multiple requests at the same time, which is efficient
HTTP GET and POST
GET requests are usually used to query and get data, while POST requests are used to send data GET requests in URLs, so GET requests must never be used to transmit sensitive data, while POST requests are in the request header with slightly higher security. Request for GET
Ps : The data requested by POST is also stored in the request header in plain text, so it is not safe.
Cookie
The World Wide Web uses cookies to track users, indicating the status information passed between the HTTP server and the user.
How cookies work:
1. The user browses a website, the server of the website generates a unique identification code for the user, and uses this as an index to generate an item in the server back-end database. 2. Add a "Set-" to the HTTP response message returned to the user. Cookie", the value is the identifier, such as 1233. The user's browser saves the cookie. There is a line of cookies for every HTTP request sent when continuing to browse the website: 123 Then, the website knows that the cookie is What did this user of 123 do, maintain a separate list for this user (like a shopping cart)
Of course, cookies are double-edged swords, which are convenient and dangerous, such as privacy leaks. Users can decide whether to use cookies at their discretion.
Session
The cookie is stored on the client, and the session is stored in the server. When the server receives the cookie sent by the user, it will find the corresponding Session according to the SessionID in the cookie. If not, a new SessionID will be generated and returned. user
All in all, cookies and Sessions are just the same thing.
HTTPS
HTTPS protocol
Based on the HTTP protocol, the HTTPS protocol adds a layer of SSL/TLS encryption layer between HTTP and TCP, which solves the problem of HTTP insecurity: impersonation, tampering, and eavesdropping.
The unique flat cable shape allows for a cleaner and safer installation. You can easily and seamlessly make the cable run along walls, follow edges&corners or even make it completely invisible by sliding it under a carpet.
Flat patch cable include cat8 flat patch cable, cat7 flat patch cable, cat6 flat patch cable and CAT5E Flat Patch Cable.
CAT8 patch cable Support bandwidth up to 2000MHz & transmitting data at speeds of up to 40Gbps,connect to LAN/WAN segments and networking gear at maximum speed.
CAT7 cable Support bandwidth up to 1000MHz & transmitting data at speeds of up to 10Gbps,connect to LAN/WAN segments and networking gear at maximum speed.
CAT6 Flat Ethernet Cable provides high performance, as it uses 250Mhz to transfer data up to 1Gbps, faster than CAT5E. Allowing for greater bandwidth and, therefore, home and office productivity.
CAT5e cable Support bandwidth up to 100MHz & transmitting data at speeds of up to 1000bps,connect to LAN/WAN segments and networking gear at maximum speed.
Flat Cable,Flat Extension Cord,Flat Ribbon Cable,Flexible Flat Cable
Shenzhen Kingwire Electronics Co., Ltd. , https://www.kingwires.com