Charles M. Kozierok The TCP-IP Guide

Подождите немного. Документ загружается.

POP3 Update State: Mailbox Update and Session Termination Process and

Commands

Once the POP3 client has completed all the e-mail message access and retrieval transac-

tions that it needed to perform, it is done with the POP3 session—except it isn't really quite

done yet. The POP3 standard defines a final session state, the Update state, to perform

various housekeeping functions, and then terminate both the POP3 session and the under-

lying TCP connection.

The transition from the Transaction state to the Update state occurs when the POP3 client

issues the QUIT command. This command has no parameters, and serves to tell the POP3

server that the client is done and wishes to end the session. The POP3 standard lists this

command as part of its description of the Update state, though it obviously is really issued

from the Transaction state.

After the POP3 server receives the QUIT command, one thing it does is to delete any

messages that were previously marked for deletion by the DELE command in the Trans-

action state. It's interesting that the Post Office Protocol chose to implement this two-stage

deletion process. The standard doesn't describe specifically why this was done, but it

seems likely that it is a precaution to insure against accidental deletion and loss of mail.

By delaying actual deletion until the Update state, the server can verify that it has received

and processed all commands prior to the move to the Update state. It also allows the

deletion of messages to be “undone” if necessary, using the RSET command, if the user

changes his or her mind about the deletion prior to exiting the Transaction state. Finally, if

any problem occurs with communication between the client and server that causes the TCP

connection to be interrupted prematurely before the QUIT command is issued, no

messages will be removed from the mailbox, giving the client a second chance to retrieve

them in case they were not received properly.

Once the deleted messages have been removed, the server returns an acknowledgment to

the client; “+OK” if the update was successful, or “-ERR” if there was some sort of a

problem removing one or more of the deleted messages. Assuming there were no

problems, the “+OK” response will also contain a “goodbye” message of some sort

indicating that the session is about to be closed. The TCP connection between the client

and server is then torn down and the session is done.

A POP3 mail retrieval session normally lasts a few seconds or minutes, but can take many

minutes if the mailbox is large and the connection between the client and server is slow.

There is no limit on how long the client and server can be connected, as long as commands

continue to be sent by the client. A POP3 server will normally implement an inactivity timer,

however, which is customizable but must have a duration of no less than 10 minutes. If the

connection is idle for the full duration of the inactivity timer, the server assumes that the

client has experienced some sort of a problem and shuts down the connection. If this

occurs, the server doe not delete any messages marked for deletion—again, this is in order

to give the client another chance to retrieve those messages if a problem occurred getting

them the first time.

Key Concept: When the POP3 client is done with its e-mail transactions, it issues

the QUIT command. This causes the Update state to be automatically entered,

where the server performs necessary clean-up operations, including deleting any

messages marked for deletion in the Transaction state.

TCP/IP Internet Message Access Protocol (IMAP/IMAP4)

The offline mailbox access model provides the basic mail access functions that most users

need. Using the popular Post Office Protocol (POP3), a user can access his or her mailbox

and retrieve messages so they can be read on a local machine. This model has the

advantage of simplicity, but does not provide many features that are increasingly in demand

today, such as keeping track of the status of messages and allowing access from many

client devices simultaneously. To provide better control over how mail is accessed and

managed, we must use either the online or disconnected access models. The Internet

Message Access Protocol (IMAP) was created to allow these access models to be used,

providing rich functionality and flexibility for the TCP/IP e-mail user.

In this section I describe the operation of the Internet Message Access Protocol. I begin as

I usually do, with a general look at the protocol, its history, versions and standards. This

includes a discussion of its benefits compared to the simpler POP3. I describe the operation

of the current version of the standard, IMAP4, focusing on how client and server devices

communicate and showing how the IMAP session moves through a series of four states. I

describe how IMAP commands, results and responses are used and discuss the

commands that can be used at any time during an IMAP session. I then describe the

processes that are performed and commands used in each of the three main IMAP states.

Related Information: The main price that IMAP pays for having a much richer set

of functionality than POP is much more complexity. In this section, I have

described IMAP in approximately the same level of detail that I did POP in its

section. Please see the appropriate RFC documents for the full description of the protocol

and more discussion of some of its nuances, particularly the syntax of the many commands

and parameters, which would take dozens of pages to cover fully here.

IMAP Overview, History, Versions and Standards

The Post Office Protocol (POP, or POP3 after its current version) has become the most

popular protocol for accessing TCP/IP mailboxes not because of its rich functionality, but in

spite of its lack of it. POP implements the offline mail access model, where mail is retrieved

and then deleted from the server where the mailbox resides, so it can be used on a local

machine. Millions of people use POP3 every day to access incoming mail. Unfortunately,

due to the way the offline access model works, POP3 cannot really be used for much else.

As I said in my overview of mailbox access models, the online model is the one we would

use in an “ideal world”, where all of us were always connected to the Internet all the time.

Offline access is a necessity because this is not the case; most user client machines are

only connected to the Internet periodically. The transfer of mail from the server to a client

machine removes the requirement that we be online to perform mail functions, but costs us

the benefits of central mail storage on the server.

The Benefits of Centralized E-Mail Storage

You may find it counterintuitive that it could be better to have mail stored on some remote

server rather than on our local computer. The main reason is flexibility of access. One of the

biggest problems with offline access using POP3 is that mail is transferred permanently

from a central server to one client machine. This is fine so long as an individual only uses

that one machine, but what if the person has separate work and home computers, or travels

a great deal? How about if a mailbox is shared by many users? These concerns have

become more and more important in recent years.

Another issue is data security and safety. Mail servers run by Internet service providers are

usually located in professionally-managed data centers. They are carefully controlled and

monitored, and backups are done on a routine basis. Most people do not take this sort of

care with their own PCs and Macintoshes, nor do they back up their data routinely. It is thus

more likely that mail be lost when it is on a local machine than when it remains on the

server.

Of course, we still have the problem of not wanting to force users to be online all the time to

access their mail. The solution is the disconnected mailbox access model, which marries

the benefits of online and offline access. Mail is retrieved for local use as in the offline

model, so the user does not have to be connected to the server continuously. However,

changes made to the mailbox are synchronized between the client and the server. The mail

remains on the server where it can be accessed from a different client in the future, and the

server acts as a permanent “home base” for the user's mail.

Recognizing these benefits, some attempts were made to implement POP using the discon-

nected access model. Typically, this was done by using POP commands to retrieve mail but

still leave it on the server, which is an option in many client programs. This works, but only

to a limited extent; for example, keeping track of which messages are new or old becomes

an issue when they are both retrieved and left on the server. POP simply lacks the features

required for proper disconnected access, because it was not designed for it.

IMAP Features

In the mid-1980s, development began at Stanford University on a new protocol that would

provide a more capable way of accessing user mailboxes. The result was the Interactive

Mail Access Protocol, later renamed the Internet Message Access Protocol (IMAP). IMAP

was designed for the specific purpose of providing flexibility in how users access e-mail

messages. It in fact can operate in all three of the access modes: online, offline and discon-

nected access. Of these, it is the online and disconnected access modes that are of interest

to most users of the protocol; offline access is similar to how POP works.

IMAP allows a user to do all of the following:

☯ Access and retrieve mail from a remote server so it can be used locally while retaining

it on the server.

☯ Set message flags so that the user can keep track of which messages he or she has

already seen, already answered, and so on.

☯ Manage multiple mailboxes and transfer messages from one mailbox to another. You

can organize mail into categories, which is useful for those working on multiple

projects, those who are on various mailing lists and so forth.

☯ Determine information about a message prior to downloading it, to decide whether or

not to retrieve it.

☯ Download only portions of a message, such as one body part from a MIME multipart

message. This can be quite helpful in cases where large multimedia files are

combined with short text elements in a single message.

☯ Manage documents other than e-mail. For example, IMAP can be used to access

Usenet messages.

Of course, there are some disadvantages to IMAP, but not many. One is that it is more

complex, but it's really not that complex, and the protocol has been around for enough

years that this is not a big issue. The most important sticking point with IMAP is simply that

it is used less commonly than POP, so providers that support it are not as easy to find as

those that support POP. This is changing, however, as more people discover IMAP's

benefits.

Key Concept: The Post Office Protocol is popular because of its simplicity and long

history, but POP has few features and normally only supports the rather limited

offline mail access method. To provide more flexibility for users in how they access,

retrieve and work with e-mail messages, the Internet Message Access Protocol (IMAP) was

developed. IMAP is primarily used in the online and disconnected access models; it allows

users to access mail from many different devices, manage multiple mailboxes, select only

certain messages for downloading, and much more. Due to its many capabilities, it is

growing in popularity.

IMAP History and Standards

IMAP has had a rather interesting history. I mean “interesting” in the sense that the normal

orderly development process that is used for most TCP/IP protocols broke down. The result

wasn't quite as bad as what occurred with SNMP, but is still unusual.

The first version of IMAP formally documented as an Internet standard was IMAP version 2

(IMAP2) in RFC 1064, published July 1988. This was updated in RFC 1176, August 1990,

retaining the same version number. However, it seems that some of the people involved

with IMAP were not pleased with RFC 1176, so they created a new document defining

version 3 of IMAP (IMAP3): RFC 1203, published in February 1991. This is described by its

authors as a “counter proposal”.

For whatever reason, however, IMAP3 was never accepted by the marketplace. Instead,

people kept using IMAP2 for a while. An extension to the protocol was later created, called

IMAP2bis, which added support for MIME to IMAP. This was an important development due

to the usefulness of MIME, and many implementations of IMAP2bis were created. Despite

this, for some reason IMAP2bis was never published as an RFC. This may have been due

to the problems associated with the publishing of IMAP3.

Note: “bis” is a Latin word meaning “again”. It is sometimes used to differentiate

changed technical documents from their previous versions when no “official” new

version number is allocated.

In December 1994, IMAP version 4 (IMAP4) was published in two RFCs: RFC 1730

describing the main protocol, and RFC 1731 describing authentication mechanisms for

IMAP4. IMAP4 is the current version of IMAP and the one widely used today. It continues to

be refined; the latest specific version is actually called version 4rev1 (IMAP4rev1), defined

in RFC 2060 and then most recently by RFC 3501. Most people still just call this “IMAP4”

and that's what I will do in the rest of this section.

IMAP General Operation, Client/Server Communication and Session States

IMAP does the same basic job as POP: it enables access to a mailbox stored on a remote

server. For this reason, it's not too surprising that IMAP is somewhat similar to POP in

operation, and you'll notice those similarities in this topic if you've already read about POP3.

However, IMAP has many more features and capabilities than POP. Therefore, it's also not

too surprising that IMAP would be more complex than POP in terms of how it operates, and

I think you will notice this as well.

Client/Server Communication

IMAP4 is a standard client/server protocol like POP3 and most other TCP/IP application

protocols. For the protocol to function, an IMAP4 server must be operating on the server

where user mailboxes are located. Again, as with POP3, this need not necessarily be the

same physical server as the one that provides SMTP service. The mailbox must in some

way be made accessible to both SMTP for incoming mail and to IMAP4 for message

retrieval and modification. A mechanism for ensuring exclusive access to avoid interference

between the various protocols is also needed.

IMAP4 uses the Transmission Control Protocol (TCP) for communication. This ensures that

all commands and data are sent reliably and received in the correct order. IMAP4 servers

listen on well-known port number 143 for incoming connection requests from IMAP4 clients.

After a TCP connection is established, the IMAP4 session begins.

Session States

The session between an IMAP4 client and server is described in the IMAP standards using

a finite state machine (FSM), a descriptive tool that I explain in the TCP operational

overview. Again, this is similar to how the operation of POP3 is explained, except that

IMAP4 is a bit more complex. Its FSM defines four states instead of three, and where a

POP3 session is linear (only going through each state once) in IMAP4 the session is not.

However, the state flow is still fairly straight-forward, mostly following a logical sequence

from one state to the next. The IMAP FSM is illustrated in Figure 309.

The following are the IMAP states, in the usual sequence in which they occur for a session:

Figure 309: Internet Message Access Protocol (IMAP) Finite State Machine

The IMAP finite state machine is slightly more complex than that of POP (Figure 306) but still rather straight-

forward. Once the TCP connection is made between client and server, the Not Authenticated state is entered;

after successful authorization, the session moves to the Authenticated state. The session may move between

Authenticated and Selected several times, as different mailboxes are selected for use and then closed when

no longer needed. From any state the session may be terminated, entering the Logout state.

Logout

(No Connection)

Not Authenticated

Authenticated

Sele cte d

Successful Authentication

Establish TCP Connection

Delete Marked Messages,

Release Resources,

Terminate Connection

Pre-Authentication

Mailbox Selection Mailbox Closed

Client LOGOUT

Or Server Close

Unsuccessful

Authentication

Or Server Close

Client LOGOUT

Or Server Close

1. Not Authenticated State: The session normally begins in this state after a TCP

connection is established, unless the special IMAP preauthentication feature has been

used (we’ll get to this feature shortly). The client at this point cannot really do much

aside from providing authentication information so it can move to the next state.

2. Authenticated State: The client has completed authentication, either through an

authentication process in the prior state or through preauthentication. The client is now

allowed to perform operations on whole mailboxes. The client must select a mailbox

before individual message operations are permitted.

3. Selected State: After a mailbox has been chosen, the client is allowed to access and

manipulate individual messages within the mailbox. When the client is done with the

current mailbox it can close it and return to the Authenticated state to select a new one

to work with, or can log out to end the session.

4. Logout State: The client may issue a Logout command from any of the other states to

request that the IMAP session be ended. The session may also enter this state if the

session inactivity timer expires. The server sends a response and the connection is

terminated.

Of the four IMAP states, only the first three are “interactive”, meaning states where

commands are actively issued by the client and responses provided by the server. Some

IMAP commands can be used while the session is in any state, while others are state-

specific. The next topic describes IMAP commands and responses in general terms.

Key Concept: IMAP is a client/server application, and an IMAP session begins with

the client making a TCP connection to the server. The session then normally starts in

the Not Authenticated state and remains there until successful authentication. In the

Authenticated state, the client may perform operations on whole mailboxes, but a mailbox

must be selected to transition to the Selected state where individual messages can be

manipulated. The client can work with many mailboxes by selecting each one in turn; it then

logs out from the server.

Normal Session Establishment and Greeting

The server determines in which state the IMAP session begins, and sends a greeting

message to tell the client the session is established and indicate which state it is in.

Normally, the server will begin the session in the Not Authenticated state. This is conveyed

to the client with the normal OK greeting message, such as this:

* OK <server-name> server ready

Preauthentication

In certain circumstances, a server may already know the identity of the client, perhaps as a

result of some external authentication mechanism not part of the IMAP protocol. In this

case, a special greeting is used:

* PREAUTH <server-name> server ready, logged in as <user-name>

This tells the client that it is already in the Authenticated state.

If the server decides for whatever reason not to accept a new session from the client, it can

respond with a BYE response instead of OK or PREAUTH, and close the TCP connection.

IMAP Commands, Results and Responses

Once an IMAP session is established, all communication between the client and server

takes place in the form of commands sent by the client and responses returned by the

server. Like POP3, commands and responses are sent as strings of ASCII text and termi-

nated with a “CRLF” sequence, making them compatible with the way data is sent using the

Telnet Protocol. However, IMAP does a few things quite differently than POP and many

other TCP/IP application protocols.

The first interesting thing about IMAP commands is that most are not abbreviated into

codes of three or four letters—they are spelled out in full. So, where POP3 has a STAT

command, the one in IMAP is called STATUS. Commands are normally shown in upper

case, as I do in this Guide, but are in fact case-insensitive.

Command Tagging and the Multiple Command Feature

IMAP also uses an interesting system of command tagging to explicitly match client

commands with certain server responses. Each time a client sends a command, it prefixes

it with a tag that is unique for the particular session. The tags are usually short strings with a

monotonically increasing number in them; the examples in the IMAP standards have the

first command tagged “a0001”, then the second “a0002” and so on. That said, as long as

each command is uniquely labelled, it doesn't matter what tagging scheme is used. When

the server needs to send a response that is specific to a command, it tags the reply with the

appropriate command tag. Not all replies are tagged, however.

The standard doesn't state explicitly why this tagging scheme is needed, but I believe it is

probably related to IMAP's multiple command feature. IMAP clients are allowed to send a

sequence of commands to the server to be processed, rather than only sending commands

one at a time. This can improve performance when certain commands would take a long

time to complete. The only restriction is that the commands must be independent enough

that the result of executing them all would be the same regardless of the order in which they

were processed. For example, sending a command to read a particular entity in combi-

nation with a command to store a value into the same entity is not allowed.

Key Concept: IMAP tags its commands with a unique identifier. These tags can then

be used in replies by the server to match replies with the commands to which they

correspond. This enables multiple commands to be sent to an IMAP server in

succession.

Command Groups

IMAP commands are organized into groups based on which session states the IMAP

session may be in when they are used. These groups are:

☯ “Any State” Commands: A small number of commands that can be used at any time

during an IMAP session.

☯ Not Authenticated State Commands: Commands that can be used only in the Not

Authenticated state. They are usually used for authentication, of course.

☯ Authenticated State Commands: These commands are used to perform various

actions on mailboxes. (Note that despite the name, these commands can also be used

in the Selected state.)

☯ Selected State: A set of commands for accessing and manipulating individual

messages that can be used only in the Selected state.

The reason for having the distinct Authenticated and Selected states and command groups

is that IMAP is designed specifically to enable the manipulation of multiple mailboxes. After

the session starts and the client is authenticated, the client is allowed to issue commands

that work with entire mailboxes. However, it may not issue commands that manipulate

individual messages until it tells the server which mailbox it wants to work with, which puts it

in the Selected state. The client can also issue mailbox commands from the Selected state.

Note: In addition to these four groups, the standard also defines an extension

mechanism that allows new commands to be defined. These must begin with the

letter “X”.

"Any State" Commands

Table 260 describes the IMAP “any state” commands, which can be used whenever

needed.

Table 260: IMAP “Any State” Commands (Page 1 of 2)

Command

Parame-

ters

Description

CAPABILITY None Asks the server to tell the client what capabilities and features it supports.