Charles M. Kozierok The TCP-IP Guide

Подождите немного. Документ загружается.

Note: “Cookie” may seem an odd term, but is used in a few contexts to refer to a

small piece of significant data—another example is found in the BOOTP/DHCP

message format. Today, most knowledgeable Web users would blink at you if you

mentioned the “HTTP state management mechanism” but they usually know what “cookies”

are.

Issues with Cookies

Cookies sound like a great idea, right? Well they are—and they aren’t. On the positive side,

cookies are absolutely essential for many of the applications that make the Web the power-

house it is today, the examples of online shopping and discussion forums being only two of

many. Most of the time, cookies are used for these sorts of useful and benign purposes.

Unfortunately, like every good idea, there are those who have found a way to turn cookies

to the “dark side” by finding ways to abuse them. There can even be potential problems with

them when there is no nefarious intent. For this reason, cookies are rather controversial.

Some of the issues with cookies:

☯ Transmission of Sensitive Information: Suppose you use an online banking

system. You log in to the server, which then stores your login and password (which

controls access to your account) in a cookie. If the application is not implemented

carefully, the message containing that cookie could be intercepted, giving someone

access to your account. Even if it is not, someone knowledgeable who gained access

to your computer could retrieve the information from the file where cookies are stored.

☯ Undesirable Use of Cookies: In theory, cookies should be a help to the user, not a

hindrance. However, any server can set a cookie for any reason. In some cases, a

server could set a cookie for the purpose of tracking the sites that a user visits, which

some people consider a violation of their privacy. Since some Web browsers do not

inform the user when a cookie is being set, they may not even be aware that this is

happening.

☯ Third-Party or “Unintentional” Cookies: While most people think of cookies as

being set in the context of a resource they specifically request, a cookie may be set by

any server to which a request is sent, whether the user realizes it or not. Suppose you

send a request to “http://www.myfavoritesite.com/index.htm” and that page contains a

reference to a tiny image that is on the server “http://www.bigbrotherishere.com”. The

second site can set a cookie on your machine even though you never intended to visit

it at all. This is called a “third-party cookie”.

Managing Cookie Use

The RFCs describing the cookie state management technique deal extensively with these

and other issues, but there is no clear-cut resolution to these concerns. Like most security

and privacy matters, the most important determinant of how significant potential cookie

abuse may be is your own personal comfort level. Millions of people browse the Web every

day letting any and all sites set whatever cookies they want, and never have a problem.

Others consider cookies an offensive idea and disable all cookies, which eliminates the

privacy concerns but can cause problems with useful applications like interactive Web sites.

As usual, the best approach is usually something in the middle, where you choose when

and how you will allow cookies to be set.

The degree to which “cookie control” is possible depends greatly on the quality and feature-

set of your Web client software. Many browsers do not provide a great deal of control in

how and when cookies are set, where others are much better in this regard. Some allow

cookies to be disabled, but come with them turned on by default, and since many people

are not even aware of the issues I have mentioned above, they will not even realize when

cookies are being sent. Most notable in this regard is the most popular browser in the world,

Microsoft’s Internet Explorer, which normally comes set by default to accept all cookies

without complaint or even comment.

Internet Explorer does allow you to disable cookies, but you have to do it yourself. It also

allows you to differentiate between first-party and third-party cookies, but again, you must

turn this on. Other browsers have more sophisticated settings, which will let you dictate

conditions under which cookies may be set and others when they may not. Some will even

let you allow certain sites to send cookies while prohibiting them from others. Better ones

will also you to visually inspect cookies, and selectively clear the ones you do not want on

your machine.

Third-party cookies can be used by online advertising companies and others to track the

sites that a Web user visits. For this reason, they are considered by many people to fall into

the general category of undesirable software called spyware. There are numerous tools

that will allow you to identify and remove tracking cookies from your computer; many are

available free on the Web.

Key Concept: HTTP is an inherently stateless protocol, because a server treats

each request from a client independently, forgetting about all prior requests. This

characteristic of HTTP is not an issue for most routine uses of the World Wide Web,

but is a problem for interactive applications such as online shopping where the server

needs to keep track of a user’s information over time. To support these applications, most

HTTP implementations include an optional feature called state management. When

enabled, a server sends to a client a small amount of information called a cookie, which is

stored on the client machine. The data in the cookie is returned to the server with each

subsequent request, allowing the server to update it and send it back to the client again.

Cookies thus enable a servers to remember user data between requests. However, they

are controversial, because of certain potential privacy and security concerns related to their

use.

Gopher Protocol (Gopher)

Let’s suppose that I told you I was going to describe a TCP/IP application layer protocol

designed for the specific purpose of distributed document search and retrieval. This

protocol uses a client/server model of operation, where servers provide links to related

resources such as files or programs that users access with client software that displays

options for the user to select. You might think that I was talking about the World Wide Web,

and for good reason. However, in this case I am actually talking about one of the Web’s

predecessors: the Gopher Protocol.

A good place to start our discussion of this protocol is with its name, which is well-chosen

for a number of reasons. The Gopher Protocol was developed at the University of

Minnesota, whose sports teams are called the “Golden Gophers” (Minnesota is known as

the “Gopher State”). This is the direct origin of the name, but it is also appropriate because

the rodent that shares it is known for burrowing, just as the protocol is designed to “burrow”

through the Internet. And of course, the term “gopher” also applies to a person who

performs errands, such as retrieving documents (they “go fer” this, and “go fer” that). ☺

The Gopher Protocol was developed in the late 1980s to provide a mechanism for

organizing documents for easy access by students and faculty at the university. The core

principle that guided the development of the system was simplicity. Gopher is designed on

the basis of a small number of core principles, and uses a very straight-forward mechanism

for passing information between client and server devices. It is described in RFC 1436,

published in March 1993.

Information Storage on Gopher Servers

Information accessible by Gopher is stored as files on Gopher servers. It is organized in a

hierarchical manner similar to the file system tree of a computer such as a Windows PC or

UNIX workstation. Just as a file system consists of a top-level directory (or folder) that

contains files and subdirectories (subfolders), Gopher servers present information as a top-

level directory that contains resources such as files, and/or subdirectories containing

additional resources. Resources on different servers can be linked together using by having

them mentioned in each others’ resource hierarchies. It is also possible for “virtual”

resources to be created that act as if they were files, such as programs that allow Gopher

servers to be searched.

Gopher Client/Server Operation

Typical use of Gopher begins with a user on a client machine creating a TCP connection to

a Gopher server using well-known TCP port

number 70. After the connection is established,

the server waits for the client to request a particular resource by sending the server a piece

of text called a selector string. Often, when a user first accesses a server, he or she does

not know what resource to request, so a null (empty) selector string is sent. This causes the

server to send back to the client a list of the resources available at the top (root) directory of

the server’s file system tree.

Directory Listing Structure

A directory list sent by the server consists of a set of lines, each of which describes one

available resource in that directory. Each line contains the following elements, each of

which is separated by a “<Tab>” character:

☯ Type Character and Resource Name: The first character of the line tells the client

software what sort of resource the line represents. The most common type characters

are “0” for a file, “1” for a subdirectory and “7” for a search service. The rest of the

characters up to the first “<Tab>” contain the name of the resource to be presented to

the user.

☯ Selector String: The string of text to be sent to the server to retrieve this resource.

☯ Server Name: The name of the server where the resource is located.

☯ Server Port Number: The port number to be used for accessing this resource’s

server; normally 70.

Each line ends with a “<CR><LF>” character sequence consistent with the Telnet Network

Virtual Terminal (NVT) specification. Upon sending the directory listing (or any other

response) the connection between the client and server is closed.

Menus and User Resource Selection

After receiving this sort of directory list, the Gopher client software will display a menu to the

user containing all the resource names the server provided. The user then selects his or her

desired item from the menu, and the client retrieves it by making a connection to the appro-

priate server and port number and sending the selector string of that resource. If this itself

represents a subdirectory, a new directory listing for that subdirectory will be sent by the

server; if some other type of resource, it will be accessed according to the requirements of

the resource type.

For example, suppose this line were sent from the server to the client:

0Gopher Introduction<Tab>intro<Tab>gopher.someserver.org<Tab>70

This would be presented to the user as the file called “Gopher Introduction” in a menu

containing other options. If the user chose it, the client would initiate a connection to the

gopher server “gopher.someserver.org” at port 70, and then send the selector string “intro”

to that server to retrieve the document.

Important Differences Between Gopher and The Web

As I hinted at the start of this discussion, both Gopher and the World Wide Web are

intended for the same basic purpose: providing access to repositories of information, with

links between related documents and resources. However, they take a very different

approach to how that information is accessed, especially in two key areas: user interface

and resource linking.

Gopher’s presentation to the user is entirely oriented around its hierarchical file system. As

a result, Gopher is inherently menu-based, and the user interface usually based on a

simple text presentation of those menus. In contrast, information on World Wide Web

servers can be organized in any manner, and presented to the user in whatever form or

fashion the owner of the server desires. The Web is much more “free form”, and there is no

need to use a directory structure unless that is advantageous in some way.

Linking in the World Wide Web is done directly between documents, most often using

Hypertext Markup Language (HTML) tags. When someone writing document A mentions

something relevant to document B, he or she puts a link to B right into document A itself.

Gopher, on the other hand, is not designed to use links in this way. Instead, linking is

intended to be done using the directory tree we described earlier.

Gopher's Role in the Modern Internet

There are some people who believe that Gopher is technically superior to the Web in a

number of respects. They consider it cleaner to have linking be done by servers, rather than

having links embedded in documents. An argument can also be made that the text orien-

tation of Gopher is efficient, better able to ensure compatibility between platforms, and also

more suited to special needs situations such as low-bandwidth links and access by those

with visual impairment. Some Gopher enthusiasts thus consider it to be a “purer” hypertext

system than the World Wide Web.

However, history shows us that despite Gopher predating the World Wide Web, the Web

overtook it in popularity in only a few short years. Today, the Web is the 900 pound gorilla of

the Internet, while most people have never even heard of Gopher. What happened?

I believe the main reason why Gopher lost out to the Web is that the Web is far more

flexible. Gopher’s use of text hyperlinks and server directory structures may be efficient, but

it is limiting. In contrast, the Web allows information to be presented in a wide variety of

ways. The open, unstructured nature of the Web made it an ideal vehicle for the creativity of

information providers and application developers. In the mid-1990s, the Web was also

perfectly poised to support the transition of computing from text to graphics, and Gopher

really was not.

Simply put, you can do more with the Web than you can with Gopher, and more people care

about functionality and breadth of options than straight efficiency. Once the Web started to

gain momentum, it very quickly snowballed, as I discuss in the topic on the Web’s history. It

took only a couple of years before Web use was well-entrenched, and Gopher was unable

to compete.

For its part, the University of Minnesota likely hastened Gopher’s demise with its contro-

versial decision to charge licensing fees to companies that wanted to use Gopher for

commercial purposes. I do not believe there was anything nefarious about this: the

university was on a limited budget and wanted companies that could afford it, to pay a small

fee to support development of Gopher software. However, computing history has shown

time and time again that there is no faster way to kill a protocol or standard than to try to

charge licensing or royalty fees for it, no matter what the reason.

By the late 1990s, Gopher was well on its way to obsolescence. As use of the protocol

dwindled, many organizations could no longer justify the cost of continuing to run Gopher

servers. Even the University of Minnesota itself eventually shut down its own Gopher

servers due to low utilization. The final “nail in the coffin” for Gopher occurred in 2002, when

a security vulnerability related to Gopher was discovered in Internet Explorer, and Microsoft

chose to simply remove Gopher support from the product rather than fix the problem.

Today, Gopher is still around, but is a niche protocol used only by a relatively small group of

enthusiasts and a handful of organizations that have a past history of using it.

Key Concept: The Gopher Protocol is a distributed document search and retrieval

protocol that was developed at the University of Minnesota in the late 1980s.

Resources are stored on Gopher servers, which organize information using a hierar-

chical directory structure. Gopher clients access servers to retrieve directory listings of

available resources, which are presented to the user as a menu from which an item may be

selected for retrieval. Gopher’s chief advantage is simplicity and ease of use, but it lacks

flexibility in presentation and the ability to effectively present graphics and multimedia. For

this reason, despite Gopher predating the World Wide Web, the Web has almost entirely

replaced it, and Gopher is now a niche protocol.

TCP/IP Interactive and Remote Application Protocols

When it comes to TCP/IP applications, file and message transfer applications such as

electronic mail, FTP and the World Wide Web get the most attention, because they are the

ones used most often on modern internetworks. Another category of TCP/IP application

protocols that is less-well-known is the group that allows users to interactively access and

use other computers directly over an internetwork, such as the public Internet. These appli-

cations are not often employed by end-users today but are still important—both from a

historical perspective, and because of their usefulness in certain circumstances, especially

to network administrators.

In this section, I provide a brief description of the “classical” interactive and remote appli-

cation protocols used in TCP/IP. The first subsection describes the Telnet Protocol, one of

the earliest and most important application protocols in TCP/IP, still used today both for

remote device access and as the basis of communication of many other protocols. The

second describes the Berkeley remote access family, often called the “r” commands or

protocols because their command names begin with that letter. Finally, I provide a brief

overview of the Internet Relay Chat (IRC) protocol, the original interactive chat application

of the Internet and one still used widely today.

Telnet Protocol

In the very earliest days of internetworking, one of the most important problems that

computer scientists needed to solve was how to allow someone operating one computer to

access and use another as if he or she were connected to it locally. The protocol created to

meet this need was called Telnet, and the effort to develop it was tied closely to that of the

Internet and TCP/IP as a whole. Even though most Internet users today never invoke the

Telnet protocol directly, they use some of its underlying principles indirectly all the time.

Every time you send a piece of e-mail, use FTP to transfer a file, or load a Web page, you

are using technology based on Telnet. For this reason, the Telnet protocol can made a valid

claim to the title of the most historically important application protocol in TCP/IP.

In this section, I describe the operation of the Telnet protocol. I begin with an overview and

history of the protocol and a discussion of the standards that define it. I describe the general

operation of Telnet clients and servers and how connections are made and maintained. I

then explain the important concept of the Network Virtual Terminal (NVT), Telnet’s protocol

commands, and how interrupts are handled using Telnet’s special synch function. I

conclude with a detailed look at Telnet’s options and how they are negotiated.

Background Information: A basic comprehension of the Transmission Control

Protocol (TCP), especially its sliding window mechanism and flow control feature,

will be helpful in understanding this section.

Telnet Overview, History and Standards

The description of internetworking protocol suites such as TCP/IP is most often done from

the lower layers working upward, just as I have done in this Guide. While this makes sense

for a number of reasons—most notably, the protocols are easier to understand this way—

this does not reflect the history of how many protocol suites were developed. Applications in

fact often come first: they are defined to meet the needs of users, and the rest of the suite is

developed to enable the application to run. Telnet and TCP/IP represent a good example of

this “top-down” process.

The history of Telnet actually goes back over a decade before the modern TCP/IP protocol

suite that we know today. As I mentioned in my overview of the File Transfer Protocol

(FTP), the early developers of TCP/IP internetworking technologies identified two overall

application needs for networks to fill: enabling direct access and indirect access to

resources. FTP was created for indirect access, by allowing a user to retrieve a resource

from a remote host, use it locally, and if desired, copy it back to its source. Telnet was

designed for direct access: to let a user access a remote machine and use it as if he or she

were connected to it locally.

The Motivation for Telnet's Development

Understanding why Telnet was needed requires us to remember the nature of computing at

the time the protocol was initially developed: the late 1960s. This was well before the era of

the small personal computers that so many of us use exclusively today. All computers of

that period were large and usually shared by many users. To work on a computer, you had

to access a physical terminal connected to that machine, which was usually specially

tailored to the needs and requirements of the host.

There were two specific issues that resulted from this situation. The first was that if you had

several different computers in an organization, each user would need a separate terminal to

access each computer he or she needed to use. This was expensive and inefficient; I can

recall reading a quote from a book that compared this situation to having a room containing

a number of television sets, each of which could only display a single channel.

The second and perhaps more significant issue was the difficulty in allowing a user at one

site to access and use a machine at another. The only method at the time for accomplishing

this was to install a dedicated data circuit from the site of the computer to the site of the

user, to connect the user’s terminal to the remote machine. Again, each circuit would only

enable access to one machine. Every combination of user and computer would have

required a separate, expensive circuit to be installed and maintained.

The solution to both of these issues was to create a more general way of allowing any

terminal to access any computer. The underlying internetwork would provide the

mechanism for communicating information between computers; this became the physical

network connecting sites and the TCP/IP protocol suite connecting networks. On top of this

would run an application protocol that allowed a user to establish a session to any

networked computer and use it: the Telnet protocol.

Telnet History

Telnet was the first application protocol demonstrated on the fledgling ARPAnet, in 1969.

The first RFC specifically defining Telnet was RFC 97, First Cut at a Proposed Telnet

Protocol, published in February 1971. Development of Telnet continued throughout the

1970s, with quite a number of different RFCs devoted to revisions of the protocol and

discussions of issues related to it. It took many years to refine Telnet and resolve all the

difficulties that were associated with its development.

The final version of the protocol, Telnet Protocol Specification, was published as RFC 854

in May 1983. Over the years other RFCs have been published to clarify the use of the

protocol and address various issues such as authentication. There are also a number of

other RFCs that define Telnet options, as we will see later in this section.

Fundamental Telnet Concepts

At first glance, it may be surprising that Telnet took so long to develop, because in theory, it

should be a very simple protocol to define: all it needs to do is send keystrokes and

program output over the network like any other protocol. This would be true if every terminal

and computer used the same communication method, but they do not. Telnet becomes

complicated because it needs to allow a terminal from one manufacturer to be able to talk to

a computer that may use a very different data representation.

Telnet solves this problem by defining a method that ensures compatibility between terminal

types and computers while allowing special features to be used by computers and terminals

that agree to support them. The protocol is built upon a foundation of three main concepts:

☯ The Network Virtual Terminal (NVT): Telnet defines a standardized, fictional terminal

called the Network Virtual Terminal (NVT) that is used for universal communication by

all devices. A Telnet client takes input from a user and translates it from its native form

to the NVT format to send to a Telnet server running on a remote computer; the server

translates from NVT to whatever representation the computer being accessed

requires. The process is reversed when data is sent from the remote computer back to

the user. This system allows clients and servers to communicate even if they use

entirely different hardware and internal data representations. Special Telnet

commands are interspersed with the data to allow the client and server devices to

perform various functions needed to manage the operation of the protocol.

☯ Options and Option Negotiation: Having Telnet clients and servers act as NVTs

avoids incompatibilities between devices, but does so by stripping all terminal-specific

functionality to provide a common base representation that is understood by everyone.

Since there are many cases where more intelligent terminals and computers may wish

to use a more advanced communication feature or service, Telnet defines a rich set of

options and a mechanism by which a Telnet client and server can negotiate their use.

If the client and server agree on the use of an option it can be enabled; if not, they can

always fall back on the NVT to ensure basic communication.

☯ Symmetric Operation: While Telnet is a client/server protocol, it is specifically

designed to not make assumptions about the nature of the client and server software.

Once a Telnet session is established, they can each send and receive data as equals.

They can also each initiate the negotiation of options. This makes the protocol

extremely flexible, and has led to its use in a variety of places, as we will discuss in a

moment.

Telnet Applications

Telnet is most often associated with remote login, which is its most common traditional use.

A user typically uses a Telnet client program to open a Telnet connection to a remote server,

which then treats the Telnet client like a local terminal, allowing the user to log in and

access the server’s resources as if he or she were using a directly-attached terminal. Telnet

is still used this way quite extensively by UNIX users, who often need to log in to remote

hosts from their local machines; I myself use Telnet in this manner every day to access a

machine hundreds of miles away. However, this use of Telnet is not nearly as common

amongst the majority of Internet users who work on Windows or Apple computers where

network resources are accessed not through direct login but by other means.

However, while remote login is a big part of what Telnet is about, it’s important to note that

the protocol was not inherently designed for that specific function. When Telnet is used to

access a remote device, the protocol itself is only used to set up the connection between

the client and server machines, encode data to be transmitted according to the rules of the