Charles M. Kozierok The TCP-IP Guide

Подождите немного. Документ загружается.

In HTTP/1.0, each TCP connection involves only one such exchange, as shown in Figure

315; in HTTP/1.1, multiple exchanges are possible, as we'll see in the next topic. Note also

that the server may in some cases respond with one or preliminary responses prior to

sending the full response. This may occur if the server sends a preliminary response using

the “100 Continue” status code prior to the “real” reply. See the topic on HTTP status codes

for more information.

Key Concept: HTTP is a client/server-oriented, request/reply protocol. Basic

communication consists of an HTTP Request message sent by an HTTP client to an

HTTP server, which returns an HTTP Response message back to the client.

Intermediaries and The HTTP Request/Response Chain

The simple request/response pair between a client and server becomes more complex

when intermediaries are placed in the virtual communication path between the client and

server. These are devices such as proxies, gateways or tunnels that are used to improve

performance, provide security or perform other necessary functions for particular clients or

servers. Proxies are particularly commonly used on the Web, because they can greatly

improve response time for groups of related client computers.

When an intermediary is involved in HTTP communication, it acts as a “middleman”. Rather

than the client speaking directly to the server and vice-versa, they each talk to the interme-

diary. This allows the intermediary to perform functions such as caching, translation,

aggregation, or encapsulation. For example, consider an exchange through a single inter-

mediary device. The two-step communication process above would become four steps:

1. Client Request: The HTTP client sends a request message to the intermediary

device.

2. Intermediary Request: The intermediary processes the request, making changes to it

if necessary. It then forwards the request to the actual server.

3. Server Response: The server reads and interprets the request, takes appropriate

action and then sends a response. Since it received its request from the intermediary,

its reply goes back to the intermediary.

Figure 315: HTTP Client/Server Communication

In its simplest form, HTTP communication consists of an HTTP Request message sent by a client to a server,

which replies with an HTTP Response.

HTTP Client

HTTP Server

Request

Response

4. Intermediary Response: The intermediary processes the request, again possibly

making changes, and then forwards it back to the client.

As you can see, the intermediary acts as if it were a server from the client's perspective,

and as a client from the server's viewpoint. Many intermediaries are designed to be able to

“intercept” a variety of TCP/IP protocols, by “posing” as the server to a client and the client

to a server. Most protocols are unaware of the existence of the interposition of an interme-

diary in this fashion. HTTP, however, includes special support for certain intermediaries

such as proxy servers, providing headers that control how intermediaries handle HTTP

requests and replies.

It is possible for two or more intermediaries to be linked together between the client and

server. For example, the client might send a request to intermediary 1, which then forwards

to intermediary 2, which then talks to the server; see Figure 316. The process is reversed

for the reply. The HTTP standard uses the phrase request/response chain to refer collec-

tively to the entire set of devices involved in an HTTP message exchange.

Key Concept: The simple client/server operational model of HTTP is complicated

when intermediary devices such as proxies, tunnels or gateways are inserted in the

communication path between the HTTP client and server. HTTP/1.1 is specifically

designed with features to support the efficient conveyance of requests and responses

through a series of steps from the client through the intermediaries to the server, and back

again. The entire set of devices involved in such a communication is called the request/

response chain.

The Impact of Caching on HTTP Communication

The normal HTTP communication model is changed through the application of caching to

client requests. Caching is employed by various devices on the Web to store recently-

retrieved resources so they can be quickly supplied in reply to a request. The client itself will

Figure 316: HTTP Request/Response Chain Using Intermediaries

Instead of being connected directly, an HTTP client and server may be linked using one or more intermediary

devices such as proxies. In this example, two intermediaries are present. The HTTP Request sent by the

client will actually be transferred three times: from the client to the first intermediary, then to the second, and

finally to the server. The HTTP Response will likewise be created once but transmitted three distinct times.

The full set of devices participating in the message exchange is called the request/response chain.

HTTP Client

Intermediary #1 HTTP ServerIntermediary #2

Request

Response

#1 #2 #3

#3 #2 #1

cache recently-accessed Web documents so that if the user asks for them again they can

be displayed without even making a request to a server. If a request is in fact required, any

intermediary device can satisfy a request for a file if the file is in its cache.

When a cache is used, the device that has the cached resource requested returns it

directly, “short-circuiting” the normal HTTP communication process. In the example above,

if intermediary 1 has the file the client needs, it will supply it back to the client directly, and

intermediary 2 and the real Web server that the client was trying to reach originally will not

even be aware that a request was ever made; the topic on HTTP caching discusses the

subject in much more detail.

Note: Most requests for Web resources are made using HTTP URLs based on a

Domain Name System (DNS) host name. The first step in satisfying such requests

is to resolve the DNS domain name into an IP address, but this process is

separate from the HTTP communication itself.

HTTP Transitory and Persistent Connections and Pipelining

The basic HTTP communication process is a simple “two-step dance”: a client sends a

request to a server and the server replies back to the client. Since this was all that HTTP

was intended to do, the first version of the protocol was designed so that after a TCP

connection was established between the client and server, a single request/response

exchange was performed. The request satisfied, the TCP connection would then be

terminated.

Transitory Connections

These connections, called transitory due to their short-lived nature, were the only type

supported by the original HTTP/0.9, and the same model was maintained in the more

widely-deployed HTTP/1.0. The advantage of this connection model is its conceptual

simplicity; the problem with it is that it is inefficient when the client needs to make many

requests to the same server. This is often the case with modern hypertext documents,

which usually carry inline references to images and other media. A typical client request for

the home page of a Web site begins with a single request for an HTML file, but then leads to

subsequent requests for each of the other related files that go with that document.

With transitory connections, each of these requests made by the client requires a new,

distinct TCP connection to be set up between the client and server. Every connection takes

server resources and network bandwidth, so having to establish a new one for each file is

woefully inefficient. Suppose that you were having a conversation with someone whom you

needed to ask a series of questions. Now imagine that after answering each question, the

other person hung up the phone and you had to call them up again! You get the picture.

There are some people who consider the temporary nature of HTTP/0.9 and HTTP/1.0

connections to be a “design flaw” of these early versions of HTTP, but I don't think that this

is necessarily very fair. In the early days, this model of operation was really not a big issue;

it only became problematic when the use of the Web and hypertext evolved.

For the first few years of its existence, hypertext was primarily that: text. Having an HTTP

session only last long enough for one request/response was generally sufficient since the

whole resource was in one file. It was only in the 1990s that hypertext became hypermedia,

with a heavy emphasis on embedded graphics and other files. When Web pages changed

from simple text to multimedia marvels sporting dozens or even hundreds of embedded

images, the limitations of HTTP/1.0 really became obvious.

Persistent Connections

The solution to the inefficiency problem of transitory connections came in HTTP/1.1, which

allows an HTTP client and server to set up a persistent connection. The basic operation of

HTTP is not changed; the main difference is that by default, the TCP connection is kept

open after each request/response set, so that the next request and response can be

exchanged immediately. The session is only closed when the client is done requesting all

the documents it needs.

Keeping the TCP connection between an HTTP client and server alive between requests is

probably the single most important way that HTTP/1.1 improves performance over version

1.0. Clients are able to get their files more quickly because they don't have to wait for a TCP

connection before each resource is retrieved. Server load is reduced and memory use in

busy servers is conserved. Network congestion is reduced through the elimination of

unnecessary TCP handshaking segments.

Pipelining

Persistent connections offer another important performance-enhancing option to HTTP

clients: the ability to pipeline requests. Suppose the client needs to send a request for files

A, B and C to a server. Since the requests for all of these files will be sent in the same TCP

session, there is no need for the client to wait for a response to its request for A before

sending the request for B. The client can send requests “rapid-fire”, one after the other. This

also improves the efficiency of the server, which will be able to fill the requests in the order

in which they are received, as soon as it is able, without having to pause to wait for each

new request to be sent.

Key Concept: HTTP/0.9 and HTTP/1.0 only supported transitory connections

between an HTTP client and server, where just a single request and response could

be exchanged on a TCP connection. This is very inefficient for the modern Web,

where clients frequently need to make dozens of requests to a server. HTTP/1.1 operates

by default using persistent connections: once a TCP connection is established, the client

can send many requests to the server and receive replies to each in turn. This allows files to

be retrieved more quickly, and conserves server resources and Internet bandwidth. The

client can even pipeline its requests, sending the second one immediately, without having to

first wait for a reply to the first one. HTTP/1.1 still supports transitory connections for

backwards compatibility, when needed.

Complications Arising From Persistent Connections and Pipelining

The obvious advantages of persistent connections make them the default for modern HTTP

communication, but they do have one drawback: they complicate the process of sending

data from the server to the client. With transitory connections, the client knows that all of the

data it receives back from the server is in reply to the one request it sent. Once it has all the

bytes the server sent and the TCP session ends, the client knows the file is complete.

With persistent connections, and especially when pipelining is used, the server will typically

be sending one file after the other to the client, which must differentiate them. Remember

that TCP just sends data as a series of unstructured bytes; the application must take care of

specifying where the dividing points are between files. This means that persistent connec-

tions and pipelining lead to data length issues that must be specially addressed in HTTP.

To provide compatibility with older versions of the software, HTTP/1.1 servers still support

transitory connections, and will automatically close the TCP connection after one response

if they receive an HTTP/0.9 or HTTP/1.0 request. HTTP/1.1 clients may also specify that

they do not want to use persistent connections in their initial request. The next topic

describes how persistent connections are created and managed.

HTTP Persistent Connection Establishment, Management and Termination

The introduction of persistent connections greatly improved the efficiency of HTTP by

cutting out most of the overhead involved in a sequence of HTTP/1.0 requests between a

client and server pair. These connections have now become the standard for client/server

communication on the Web.

Connection Establishment

Like most TCP/IP client/server protocols, the server in HTTP plays the passive role by

listening for requests on a particular port number. The default port number for HTTP is well-

known TCP port number 80, and is used by Web browsers for most HTTP requests, unless

a different port number is specified in the URL. The client initiates an HTTP connection by

opening a TCP connection from itself to the server it wishes to contact.

Note: A DNS name resolution step may precede the entire HTTP connection,

since most URLs contain a host name, while HTTP requires that the client know

the server's IP address. This can cause confusion, especially since DNS uses

UDP where HTTP uses TCP.

Once the TCP connection is active, the client sends its first request message. As we will

see in the next section, the request specifies what version of HTTP the client is using. If this

is HTTP/0.9 or HTTP/1.0, the server will automatically work in the transitory connection

model, and will only send one reply and then close the link. If it is HTTP/1.1, the assumption

is that a persistent connection is desired. An HTTP/1.1 client can override this by including

the special Connection: close header in its initial request, which tells the server it does not

want to keep the session active after the request it is sending has been fulfilled.

Persistent Connection Use and Management

Assuming that a persistent connection is being used, the client may begin pipelining subse-

quent requests after sending its first request, while waiting for a response from the server to

the initial query. As the server starts to respond to requests, the client processes them and

takes action such as displaying the data retrieved to the user. The data received from the

server may also prompt the client to request more files on the same connection, as in the

case of an HTML document that contains references to images.

The server will generally buffer a certain number of pipelined requests from the client. In the

case where the client sends too many requests too quickly, the server may “throttle back”

the client using the flow control mechanism built into TCP. In theory, the server could also

just decided to terminate the connection with the client, but it is better for it to use TCP’s

existing features. Closing the connection will cause the client to initiate a new connection,

potentially exacerbating any overloading problem.

Connection Termination

The flow of requests and responses continues for as long as the client has requests. The

connection can be gracefully terminated by the client by including the Connection: close

header in the last request it needs to send to the server. All requests are filled in order, so

the server will satisfy all outstanding requests and then close the session.

Since HTTP/1.1 supports pipelining of requests, there is usually no need for a client to

establish more than one simultaneous connection to the same server. Clients occasionally

do this anyway to allow them to get information from a server more quickly. This is

considered by many to be “anti-social”, because it can lead to a busy server's resources

being monopolized by one client to the exclusion of others that want to access it.

Under special circumstances, either the client or the server may unexpectedly close an

active persistent connection. For example, if the client detects that too much time has

elapsed since the server last replied, it may conclude that the server has crashed and

terminate the connection. Similarly, the server might receive a shutdown command from its

administrator or for other reasons end a session with a client abruptly. Servers normally

avoid closing down a link during the middle of sending a response.

Both clients and servers must be able to handle abrupt session termination. For servers,

there is not much to do; if the client terminates the connection the server simply “cleans up”

any resources associated with the connection, and goes on to service the next client.

Clients have more to do when a server prematurely terminates a session, and this is

especially the case when requests are pipelined. The client must keep track of all requests

sent to the server to ensure that each is filled. If the server closes the session unexpectedly,

the client will usually attempt to establish a new connection to retransmit the unfilled

requests. Since an abrupt session close is often a sign of a busy server, the HTTP standard

specifies that clients use a binary exponential backoff algorithm to wait a variable but

increasing amount of time before submitting re-requests for files (similar in concept to the

method used to deal with collisions in Ethernet). This helps prevent clients from “piling on”

requests to a device that is already overwhelmed.

HTTP Messages, Message Formats, Methods and Status Codes

As we saw in the operational overview of the Hypertext Transfer Protocol, HTTP is entirely

oriented around the sending of client requests and server responses. These take the form

of HTTP messages sent between clients and servers. As with all protocols, HTTP uses a

special format that dictates the structure of both client request messages and server

response messages; understanding how these messages work is a big part of compre-

hending HTTP as a whole.

In this section I describe the messages used by HTTP and the specific commands and

responses issued by clients and servers. I begin with a look at the generic HTTP message

format and the major components of every HTTP message. I then discuss the specific

formats used for both requests and responses. I explain the different types of HTTP

methods (commands) used in client requests, and the HTTP status codes used in server

replies.

Note: Much of the functionality of HTTP is implemented using header fields that

appear at the start of each HTTP request and response. Due the number of

headers and their potential complexity, they are covered in detail in their own

section.

HTTP Generic Message Format

As we’ve already seen, all of the communication between devices using the Hypertext

Transfer Protocol takes place via HTTP messages, of which there are only two types:

requests and responses. Clients usually send requests and receive responses, while

servers receive requests and send responses. Intermediate devices such as gateways or

proxies may send and receive both types of message.

All HTTP messages are created to fit a message structure that the standard calls the

generic message format. Like most of the other TCP/IP messaging protocols such as

SMTP and NNTP, HTTP messages do not use a binary message format; rather, they are

text-based. HTTP messages are based loosely on the electronic mail RFC 822 and 2822

message standards, as well as the Multipurpose Internet Mail Extensions (MIME) standard.

I say “loosely” because HTTP messages are similar in construction to e-mail messages but

do not strictly follow all of the e-mail or MIME format requirements. For one thing, not all of

the RFC 822 and MIME headers are used; there are other differences as well, which we will

soon examine.

The HTTP generic message format is as follows:

<start-line>

<message-headers>

<empty-line>

[<message-body>]

[<message-trailers>]

You can see that this is pretty much the same as the format used for e-mail messages and

for Usenet newsgroup messages too: headers, an empty line and then a message body. All

text lines are terminated with the standard “CRLF” control character sequence; the empty

line contains just those two characters and nothing else. The headers are always sent as

regular text; the body, however, may be either text or 8-bit binary information, depending on

the nature of the data to be sent. (This is another way that HTTP does not adhere strictly to

the RFC 822 standard; see the discussion of entities and media types for a full discussion.)

The start line is a special text line that conveys the nature of the message. In a request, this

line indicates the nature of the request, in the form of a method, as well as specifying a URI

to indicate the resource that is the object of the request. Responses use the start line to

indicate status information in reply to a request. More details on the use of the start line can

be found in the next two topics that detail request messages and response messages

respectively.

Header Format

There are many dozens of message headers defined in HTTP, which are organized into

groups by function as we will soon see. Almost all of these are optional; the one exception

is the Host header, which must be present in each request in HTTP/1.1. Headers may be

sent in any order, and all follow the same header format used in e-mail messages:

<header-name>: <header-value>

Message Body Contents

The message body is optional, because it is only needed for certain types of messages.

The body may carry a set of information to be communicated between the client and server,

such as a detailed error message in a response. More commonly, when present, it carries a

file or other resource, which is formally called an entity in the HTTP standard. Entities are

most often found in the body of a response message, since most client requests ask for a

server to send a file or other resource. However, they can also be found in certain requests.

HTTP supports many kinds of entities and provides encoding methods to support a wide

range of media. Special headers describe the characteristics of the entity, if present. HTTP

entities and related issues are described in detail in their own section of the Guide.

Key Concept: All HTTP messages conform to a structure called the generic

message format. This format is based on the RFC 822 and MIME electronic mail

message standards, though HTTP does not follow those formats precisely. Each

HTTP message begins with a start line, then contains a number of message headers,

followed by an empty line and optionally a message body. The body of the message may

contain a resource such as a file to be communicated between client and server, called an

entity.

Chunking and Message Trailers

HTTP/1.1 uses persistent connections by default, so messages are sent in a steady stream

from client to server and server to client. This requires that some means be used to mark

where one message ends and the next begins, which is usually accomplished in one of two

ways. The first is using a special header that indicates the length of the message so the

receiving device knows when the entire message has been received. The second is a

method called “chunking”, where a message is broken into pieces for transmission and the

length of each piece indicated in the message body.

When chunking is done, a set of message trailers may follow the body of the message.

Trailers are in fact the same as headers except for their position in the file, but may only be

used for entity headers. See the topic on data length issues, “chunked” transfers and

trailers for more details on all of these matters.

HTTP Request Message Format

The client initiates an HTTP session by opening a TCP connection to the HTTP server with

which it wishes to communicate. It then sends request messages to the server, each of

which specifies a particular type of action that the user of the HTTP client would like the

server to take. Requests can be generated either by specific user action (such as clicking a

hyperlink in a Web browser) or indirectly as a result of a prior action (such as a reference to

an inline image in an HTML document leading to a request for that image.)

HTTP requests use a message format that is based on the generic message format

described in the preceding topic, but specific to the needs of requests. The structure of this

format is as follows (see Figure 317):

<request-line>

<general-headers>

<request-headers>

<entity-headers>

<empty-line>

[<message-body>]

[<message-trailers>]

Request Line

The generic start line that begins all HTTP messages is called a request line in request

messages. Its has a three-fold purpose: to indicate the command or action that the client

wants performed; to specify a resource upon which the action should be taken; and to

indicate to the server what version of HTTP the client is using. The formal syntax for the

request line is: