Download A Protocol for Packet Network Intercommunication and more Schemes and Mind Maps Network Technologies and TCP/IP in PDF only on Docsity!
I E E E TR.INSACTIOSS O X COMMUNIC~LTIOKS, VOL. COM-22, NO. 5, MAY 1974
A Protocol For Packet Network Intercommunication
VINTON G. CERF AND ROBERT E. ICAHN, MEMBER, IEEE
637
Absfract-A protocol that supports the sharing of resources that exist in different packet switching networksis presented. Theproto- col provides for variation in individual network packet sizes, trans- mission failures, sequencing,flow control, end-to-end error checking, and the creation and destruction of logical process-to-process con- nections. Some implementation issues are considered, and problems such as internetwork routing,accounting, and timeouts areexposed.
INTRODUCTION
I"
1 THE LAST few years considerable efforthasbeen
expended on the design and implementation of packet
switching net\vorl<s [l]-[7],[14],[17]. A principle reason
for developing such not\vorks has been to facilitatethe
sharing of computer resources. A packet communication
network includes a transportation mechanism for dcliver- ing data between computers or between computersand terminals. To make the data meaningful, computers and tcrminals share a common protocol (i.c.,a set of agreed upon conventions). Several protocols havealready been developed for this purpose [S]-[12],[16]. However, these protocols have addressed only the problem of com- munication on the same nct\vork. I n this paper we prcscnt a protocol design and philosophy that supports thesharing of resources that exist in different packct switching net- works. Aftera brief introduction to internetwork protocol issues, we describe the function of a GATEWAY as an intcr- face bctwccn nctn-orks and discuss its role in the protocol.
We then consider thc various det,ails of the protocol,
includingaddressing, formatting, buffering, scquoncing, floxv control, error control, and so forth. Wc close with a description of an interprocess communication nxchanism and show how i t can be supported by the internet\-ork protocol. Eventhoughmany different and complex problems must be solved inthe design of an individualpacket switchingnetwork, these problems are manifestly com- poundedwhen dissimilar networks arc interconnected. Issues arise which may have no direct counterpart in an individual network and which strongly influence the way in which internetwork communication can take place. A typicalpacket switching network is composed of a
tions of the IEEE Communications Society for publication without
Paper approvedby the Associate Isditor for Data Communica- oral presentation.Manuscript received Novemtxr 5, 1973. The research reported in this paper was supported in part hy the Ad- vancedResearch Projects Agency of theDepartment of Ihfense under Contract DAHC 15-73-C-0370. trim1 Engineering, Stanford University, Stanford, Calif.
V. G. Cerf is with the Department of (^) Computer Science and Elec- It. E. Kahn is with theInformation Processing Technology Office, AdvancedResearch Projects Agency, Department of De- fense, Arlington, Va.
set of computer resources called HOSTS, a set of one or
more packetswitches, and a collcction of communication
media thatinterconnectthe packct switches. Within each HOST, wc assume thatthere exist processes which must communicate with processes in their own or other HOSTS. Any current definition of a process will be adequate for our purposes [13]. These processes are generally the ultimate source and destination of data in the network. Typically, within an individual network,there exists a protocol for communicationbetween any source and destination process. Only the source anddestination processes require kno\-ledge of this convention for com- munication to talx place. Processes in two distinct nct- works would ordinarily use different protocols forthis purpose. The ensemble of packetswitches and com- municationmedia is called the paclxt'switchingsubnet. Fig. 1 illustrates these idcas. In a typical packetswitching subnet,data of a fixed maximum size arc accepted from a source HOST, togethcr with a formatteddestinationaddress which is used to route the data ina store and forwardfashion. The transmit time for thisdata is usually dependent upon internal net\-orkparamctcrs such as communicationmedia dat>a ratcs, buffering and signaling strategies,routing, propa- gation delays, etc. In addition, somc mechanism is gen- erallyprcscnt for errorhandling anddetermination of status of the networkscomponents. Individual pacltct switching nctn;orl<s maydifferin their implementations asfollows.
- Each net\vorlt may have distinct ways of addressing the rcccivcr, thus requiring that a uniformaddressing schemc be created Tvhich can be undcrstood by each individual nctworlt.
- Each nct\vorl< may accept data of different maximum
size, thus requiring nct\vorl<s to deal inunits of the
smallestmaximum size (which may he impractically small) or requiring procedures which allow data crossing a network boundarytobcrcformattedinto smaller picccs. 3 ) The success or failure of a transmission and its pcr- formancc in eachnetwork is governed by different time dclays in accepting, delivering, and transporting the data. This requires careful development of intersetwork timing procedures to insurc thatdatacan be successfully dc- livcred through tho variousnctworlts. 4) Within each nct\vorl;, communication may be dis- ruptcddueto unrccoverahlc mStation of thedata or missing data. End-to-cnd restoration proccduros are desirable to allow complete recovery from these con- ditions.
636 IEEE TRAKSACTIONS ON COMMUNICATIONS. MAY 1074
/n PACKET-SWITCHING \ SUBNETWORK
(-) PS I PS
intact the internal operation of each individual network This is easily achieved if twonetworksinterconnect a: ifeach were a HOST to the other network, but withoul utilizingorindeedincorporating anyelaborate H O S ~ protocol transformations.
It is thus apparent that the interfacebetween network;
must play a central role in the development of any net
work interconnection strategy. We give a special name tc
this interface that performs these functions and call it : GATEWAY.
T H E GATEWAYNOTION
P A C K E T - S W i T C H I N GN E T W O R K PSSWITCH = PACKET Fig. 1. Typicalpacket switchingnetwork.
5 ) Status ,information,rout,ing,faultdetection,and isolation are typically different in each network. Thus, to obtain verification of certainconditions,suchas an in- accessible ordeaddestination,variouskinds of coordi- nation must be invoked between the communicating net- works.
It would be errtremely convenient if all the differences
betweennetworks could be economically resolved by suitableinterfacing a t .thenetworkboundaries.For many of the differences, this objective can be achieved. However, both economic and technical considerations lead us to prefer that the interface be as simple and reliable as possible and deal primarily with passing data between networks that use differentpacketswitchingstrategies. The question now arises asto whether the interface ought to account for differences in HOST or process level protocols by transforming thesource conventions into the correspondingdestinationconventions.Weobviously wantto allow convcrsion betweenpacketswitching strategies at the interface, to permitinterconnection of existing and planncd networks. However, the complcxity and dissimilarity of the Hosl7 or process level protocols makes it desirable to avoid having to transform between them at the interface,even if thistransformation were always possiblc. Rather,,compatible HOST and process levcl protocols must bc developed to achicvceffective intcrnctxork resourcc sharing. The unacceptable al- ternative is for every HOST or process to implcmcnt every protocol (a potentially unbounded number) that may be needed to cornmunicatc with other networks.Wethere- fore assume that a comnmn protocol is to be used between HOST'S or processes i n diffcrcntnetworks andthatthe interface bctn-ccn networks should takc as small a role as possiblc in this protocol. To allow nc:tworl<s under diffcrcnt ownership to inter- cunncct, somc accounting will undoubtedly be needed for traffic that passcs across the interface. Inits simplest tcrnms, this involves an accounting of packets handled by mch not for n-hich chargesarb passcd from nettonet until thc buck finally stops at the user or his rcprescnta- tivcb. Ihrthcrmorc~,the interconnection must prcserve
I n Fig. 2 we illustrate three individual networkslabelec
A , B , and C which are joined by GATEWAYS M and N
GATEWAY A// interfacesnetwork A withnetwork B, anc
GATEWAY N interfacesnetwork B to network C. W
assume that an individual network may have more t,ha~
one GATEWAY (e.g., network B ) and that there may b
more than one GATEWAY path to use in going between I pa,ir of networks. The responsibility for properly routin data resides in the GATEWAY. I n practice, a GATEWAY between two networks may b composed of twohalves,eachassociatedwith it,s ow
network. It is possible to implement each half of a GATE
WAY so it need only embed internetwork packets in loca packetformat or extractthem. We propose thatth GATEWAYS handleinternetworkpacket,sinastandarc format, but me arenot proposing anyparticulartrans mission procedure between GATEWAY halves.
Let us now trace the flow of data through the inter
connectednetworks.Weassume a packet of data fron
process X entersnetwork A destined for process Y il
network C. The address of Y is initially specified b:
process X and the address of GATEWAY M is derked fron
the address of process Y. We nmakc no attempt to spccif:
whether the choice of GATEWAY ismadeby process X
its HOST, or one of thc packet switches in network -4. Thl packet traverses network A until it reaches GATEWAY iI At the GATEWAY, the packet is reformatted to meet thl
requirements of network B, account is taken of this uni
of flow between A and B, and the GATEWAY delivers ths packettonetwork B. Again the dcrivation of the nex GATEWAY address is accomplished based on the address o
the destination Y. In thiscase, GATEWAY A T is the next one
Thc packettraverscsnetwork R until it finally rcache GATEWAY N whcrc it is formattcd tomcet the requirement
of network C. Account is again taken of this unit of f l o ~
betwccnnetworks B and C. Upon enteringnetwork C
the packet is routedtotheHosrin which process I
resides and there itis delivered to its ultimatedesbination Since the GATEWAY must understand the address of t h source anddestination HOSTS, thisinformationmust b availableina standardformatin everypacket whicl arrives at the GATEWAY..This information is containec in an internetzoork header prefixed to the packet by t h source HOST. The packetformat, including the internet
G40 IEEE TRANSACTIONS ON COMMUNICATIONS, MAY 1974
the one me suggest) are adopted,differences in cost can be used as an economic lever towardoptimization of indi- vidual network performance.
PROCESS LEVEL COMMUNICATION
We suppose that processes wish to communicate in full duplexwiththeir correspondent’s usingunbounded but
finite length messages. A single character might constitute
the text of a message from a process to a terminal or vice
versa. An entire page of characters might constitute the text of a message from a file to a process.- A data stream (e.g.,acontinuouslygeneratedbitstring)canbe repre- sented as a sequence of finite length messages. Within a HOST we assume theexistence of .a transmission controlprogram (TCP) whichhandlesthetransmission and acceptance of messages on behalf of the processes it serves. The TCP is in turn served by one or more packet switches connected to the HOST in which the TCP resides. Processes thatwantto communicatepresentmessages
to the TCPfor transmission, and TCP’Rdeliver incoming
messages totheappropriatedestination processes. We allow the TCP to break up messages into segments be- cause the destination may restrict the amountof data that mayarrive, because the local networkmaylimitthe maximumtransmission size, or because theTCPmay need toshareits resources amongmany processes con- currently.Furthermore, me constrain thelength of a segment to an integral number of 8-bit bytes. This uni- formity is mosthelpful in simplifying the software needed with HOST machines of differentnatural wordlengths. Provision at the process level can be made for padding a message that is not an integral number of bytes and for idcntifyingwhich of the arrivingbytes of text contain information of interest t o the receiving process. Multiplexinganddemultiplexing of segmentsamong processes arefundamental t.asks of the TCP. On trans- mission, a TCP must multiplextogethersegmentsfrom different source processes andproduceinternetwork packets for delivery to one of it.s serving packet switches. On reception, a TCP will accept a sequence of packets fromitsservingpacketswitch(es).Fromthis sequence of arrivingpackets(generallyfromdifferent HOSTS), the TCP must beable to reconstruct anddeliver messages to the proper destinationprocesses. We assume that every segment is augmented with ad- ditionalinformation that allows transmittingand re- ceiving TCP’s t o identify destination andsource processes, respectively.Atthispoint, we must face a major issue. How should the source TCI’ format segments destined for the same destination TCP? We consider two cases. Case 1) : If we take t.he position that segment boundaries are immaterial and that a byte stream can be formed of segments destined for the same TCP, then we may gain improvcd transmission efficiency and resource sharing by arbitrarily parceling the stream into packets, permitting manystgments t o sharea single internetworkpacket headcr.Howcver, this position results in the need to re-
construct exactly, and in order, the stream of text bytes
produced bythe source TCP. Atthedestination,this
stream must first be parsed intosegmentsandthese in turn must be used to reconstruct messages for delivery to the appropriate processes. Therearefundamental problems associated withthis strategy due to thepossible arrival of packets out of order atthe destination.Themostcritical problem appears to be the amountof interference that processes sharing the sameTCP-TCPbytestreammaycauseamongthem- selves. This is especially so at the receiving end.First, the TCP may be put tosome trouble to parse the stream back into segments and then distribute them to buffers
wheremessages are reassembled. If it is not readily ap-
parentthat all of a segmenthasarrived(remember,it may come as severalpackets),the receiving TCP may have to suspendparsingtemporarily until more packets have arrived. Second, if a packet is missing, it may not be clear whether succeeding segments, evenif they are identi- fiable, can be passed on to thereceiving process, unless the TCP has knowledge of some process level sequencing scheme. Such knowledge would permit the TCP to decide whether a succeedingsegmentcould be delivered toits waiting process. Finding the beginning of a segment when there are gaps in the byte stream mayalso be hard.
Case 2 ) : Alternatively, we might take the position that
the destination TCP should be able to determine, upon its arrival and without additional information, for which process or processes a received packet is intended, and if so, whether it should be delivered then.
If the TCPis to determine for which process an arriving
packet is intended, every packet must contain a proces header (distinct from the internetwork header) that com- pletely identifies thc destination process. For simplicity, we assume that each packet contains text from a single process which is destined for a single process. Thus each packet needcontainonlyone process header. To decide whether the arriving datais deliverable to the destination process, the TCP must be a.ble to determine whether the data is intheproper sequence(wecan make provision for the destination process to instruct its TCP to ignore sequencing, but thisis considered a special case). Withthc assumption that eacharrivingpacketcontainsa process header, the necessary sequencing and destination procesf ident)ification is immediately available to the destinatior TCP. Both Cases 1) and 2) provideforthe demultiplexing and delivery of segments todestination processes, but only Case 2 ) does so without the introductionof potential interprocess interference. Furthermore, Case 1) introduceE extramachinery to handle flow controlon a HOST-to- HOST basis! since there must also be someprovision for proccss level control, and this machineryis little used since the probability is small thatwithin a given HOST, two processes d l be coincidentally scheduled to send messages to the same destination HOST. For this reason, we select
the method of Case 2 ) as a part of the internetwork
transmission QrOtOCOl.
CERFANDKAHN:PACKETNETWORKINTISRCOMMUNICATION
ADDRESSFORMATS The selection of address formats is a problem between networksbecause the local networkaddresses of TCP's may vary substantially in format and size. A uniform in- ternetwork TCP addressspace,understood by each GATEWAY and TCP, isessential to routingand delivery of internetwork packets. Similartroublesareencountered when we dealwith process addressing and, more generally, port addressing. We .introduce the notion of ports inordertopermit a process to distinguish between multiple message streams. The port is simplya designator of one suchmessage stream associated with a process. The meansfor identifying a port are generally different in different operating systems, and therefore, to obtain uniform addressing, a standard port address format is also required. A port address designates a full duplex message stream. TCP ADDRESSING TCP addressing is intimatelyboundupinrouting issues, since a HOST or GATEWAY must choose asuitable destination HOST or GATEWAY for an outgoing int,ernetworl< packet. Let us postulate the following address format for the TCP address (Fig. 4). The choice for network identi- fication (8 bits) allows up to 256 distinct networks. This size seems sufficient for the forseeable future. Similarly, theTCP identifier field permits upto 65 536 distinct TCP's to be addressed, which seems more than sufficient for any given network. As each packet passes through a GATEWAY, the GATEWAY observes the destinationnetwork I D to determine how toroutethepacket. If the destinationnetwork is con- nected to the GATEWAY, the lower 16 bits of the TCPaddress are used to produce a local T C P address in the destination network. If the destination network is not connectedto the GATEWAY, the upper S bits are used to select a subsequent GATEWAY. We malx noeffort to specify how each in- dividualnetworkshallassociate the internetwork T C P
identifier with its local TCP address. We also do not rule
out the possibility that the local network understands the internetworkaddressingschemeandthusalleviates the GATEWAY of the routing responsibility. PORT ADDRESSING A receiving T C P is faced with the task of demultiplex- ing thestream of internetworkpackets it receives and reconstructing the original messages for each destination process. Eachoperatingsystemhasits own internal
means of identifying processes and ports. We assume that
16 bits aresufficient to serve as intcrnctwork portidentifiers. Asending process nccd not know how the destination
port identification will be used. The destination TCP
will beablc to parse thisnumberappropriately to find the proper buffer into which it will place arriving packets. We permit a large port number field to support processcs which wantto distinguishbctween many different messages streams concurrently. I n reality, we do not care how the 16 bits are sliced up by the TCP'sinvolved.
641
8 16 NETWORK TCP IDENTIFIER Fig. 4. ',TCPaddress.
Even though the transmitted port name field is large, it is still a compact external name for the internal repre- sentation of the port. The use of short names for port identifiers is often desirable to reduce transmission over- head and possibly reducepacket processing time at the dehnationTCP. Assigning short names to each port, however,requires an initialnegotiationbetweensource and destination to agree on a suitable short name assign- ment, the subsequentmaintenance of conversion tables a t both the source and the destination, and a final trans- action to release the short name. For dynamic assignment of port names, this negotiation is generally necessary in any case.
SEGMENTANDPACKETFORMATS
As shownin Fig. 5, messages are broken by the TCP
into segments whose formatis shown in moredetailin Fig. 6. The field lengths illustrated are merely suggestive. The first two fields (source port and destination port in the figure) have already been discussed in the preceding sectiononaddressing. The uses of t.he third and fourth fields (window and acknowledgmentin the figure) will be discussed laterinthe sectiononretransmission and duplicate detection.
We recall from Fig. 3 that an internetwork header con-
tains both a sequence number and a byte count, aswell as a flag field and a check sum. The USCS of these fields are explained in the following section.
REASSEMBLYANDSEQUENCING The reconstruction of a message at the receiving T C P clearly requires' that eachinternetworkpacketcarry a sequence number which is unique to its particular desti- nation port message stream. The sequence numbers must bemonotonic increasing (or decreasing) since thcyare
used to reorder and reassemble arrivingpacketsinto a
mcssage. If the space of sequence numbers were infinite, we could simply assign the next one to each new packet. Clearly, this space cannot be infinite, andwe will consider what problems a finite sequence number space will cause when we discuss retransmission andduplicatedetection in the next section. We propose the following scheme for performing the sequencing of packets and hence the re- construction of messages by the destination TCP. A pair of ports will exchange one or more messages over
a period of time. We could view the sequence of messages
producedbyone port as if it were embedded in an in- finitely longstream of bytes. Each byteof the message has a unique sequence number which we takc to be its byte location relativc to the beginning of the stream. When a
Inthe case of encryptedpackets, a preliminary stage of re- assembly may be required prior to decryption.
CRRF AND KAHX: PACKET NETWORK INTERCOMMUNICATION 643
RETRANSMISSIONANDDUPLICATE DETECTION
No transmissioncanbe 100 percent reliable. We
proposea timeout and positive acknowledgment mecha- nism which will allow TCP’s torecover from packet losses from one HOST to another. A TCP transmits packets and waits for replies (acknowledgements) that are carried in
the reverse packetstream. If noacknowledgment for a
particularpacket is received, theTCP will retransmit.
It isourexpectation that the HOST level retransmission
mechanism,which is described inthe following para- graphs, will notbe called uponveryofteninpractice. Evidence already exists2 that individual networks can be effectively constructed without this feature. However, the
inclusion of a HOST retransmissioncapabilitymakes it
possible to recover from occasional network problems and allows a wide range of HOST protocol strategies to be in- corporated. We envision it will occasionally be invoked to allow HOST accommodation to infrequent overdemandsfor limited buffer resources, and otherwise not used much. Anyretransmission policy requiressomemeans by which the receiver can detect duplicate arrivals. Even if an infinitenumber of distinct packet sequencenumbers were available, the receiver mould still have the problem of knowing how long to rememberpreviouslyreceived
packets in order to detect duplicates. Matters are compli-
cated by the fact that only a finite number of distinct sequencenumbers areinfactavailable,and if they are reused, the receiver must be able to distinguish between new transmissions and retransmissions.
A window strategy, similar to that used by the French
CYCLADES system (voie virtuelle transmission mode [SI) andthe ARPANET verydistant HOST connection [lS],
is proposed here (see Fig. 10).
Suppose that the sequence number field in the inter- network header permits sequence numbers to range from
0 to n - 1. We assume that the sender will not transmit
more than w bytes without receiving an acknowledgment. The w bytes serve as the window (see Fig. 11). Clearly, w must be less than n. The rules for sender and receiver are as follows.
Sender: Let L be the sequence number associated with
the left window edge.
- Thesendertransmitsbytesfromsegments whose text lies between L and up to L + w - 1. 2 ) On timeout(durationunspecified),thesender retransmits unacknowledged bytes. 3) Onreceipt of acknowledgment consisting of the receiver’s currentleft window edge, the sender’s,left windowedge is advancedoverthe aclrnowledged bytes (advancing the rightwindow edge implicitly). Receiver:
- Arrivingpacketsyhose sequencenumbers coincide with the receiver’s current left window edge are acknowl- edged by sending to the source the next sequence number
Left Window Edge I 0 a a+w- 1 n- 1
I< packet sequence number space^ - Fig. 10. The windowconcept.
Source Address
I Address
Destination I
6 7 8 9 10
Next Read Position End ReadPosition
Timeout
Fig. 11. Conceptual TCBformat.
expected. This effectively acknowledges bytes in between. The left windowedge is advanced to the next sequence number expected.
- Packets arriving with a sequence number to the left
of the window edge (or, in fact, outsideof the window) are
discarded, and the current leftwindow edge isreturned as acknowledgment. 3) Packets whosesequencenumbers lie withinthe receiver’s window but do not coinicide with the receiver’s leftwindowedge areoptionallykept ordiscarded, but are notacknowledged. This is the case when packets arrive out of order. We make some observations on this strategy. First, all computations with sequence numbers and window edges must be mademodulo n (e.g., byte 0 follows byte n - 1). Second, w must be less than n / Y ; otherwisea retrans- mission mayappeartothe receiver to bea new trans- mission in the case thatthe receiver hasaccepteda window’s worth of incoming packcts, but all acknowledg- ments havc been lost. Third, the receiver can either save ordiscardarrivingpackets whose!sequence numbersdo not coincide with the receiver’s left window. Thus, in the simplestimplementation,the receiver need notbuffer more than onepacketpermessagestream if space is critical. Fourth,multiplepacketscan be aclrnowledgcd simultaneously.Fifth,the receiver is ableto deliver messages t o processes in their proper order as a natural result of the reassemblymechanism. Sixth, whendupli- catesarcdetected,the acknowledgmentmethodused naturally works t o rcsynchronizcscndcr and receiver. Furthermore, if the rcccivcr acceptspackets whose sequcncenumbcrs lie withinthecurrent window but
The ARPANET is one such example. (^) required that a retransmission not appear to be a new transmission. Actually n / 2^ ismerely^ a convenient number^ t o^ use;^ it^ is^ only
644 IEEE TRANSACTIOM ON COMMUNICATIONS, MAY 1974
which are not coincident with the left window edge, an acknowledgment consisting of thecurrentleft window edge wouldact as a stimulus tocause retransmission of the unacknowledgedbytes.Finally, we mention an overlap problemwhichresultsfromretransmission, packet splitting,andalternaterouting of packetsthrough dif- ferent GATEWAYS. A600-byte packetmight pass through one GATEWAY andbebrokenintotwo300-bytepackets. On retrans- mission, the samepacketmight bebrokenintothree 200-byte packets going throughadifferent GATEWAY. Since each byte has a sequence number, there is no con- fusion at the receiving TCP. We leave for later the issue of initiallysynchronizing thesenderand receiver left window edges and the window size.
FLOWCONTROL
Every segment that arrives at the destination TCP is ultimately acknowledged byreturningthe sequence number of the next segment which must be passed to the process (it may not yet have arrived). Earlier we described the use of asequence number spaceand window to aidinduplicatedetection. Ac- knowledgments are carried in the processheader(see Fig. 6)and- along withthemthere is proviqion for a “suggested window”.which the receiver can use to control the flow of data from the sender. This is intended to be the main component of the process flow controlmecha- nism. The receiver is frcc to vary the windo& size accord-
ing toany algorithm it desires so longas the window
size never exceeds half thc sequence number space. This flow controlmechanism is exceedinglypowerful and flexible and does notsuffer fromsynchronization troubles that may be encountered by incremental buffer allocationschemes [9],[lO]. Hoivever, it relies heavily on an effective retransmission strategy. The receiver can reduce the window even while packets are en route from the senderwhosewindow is presentlylarger.Thenet effect of thisreduction will be thatthe receiver may discardincomingpackets (theymaybeoutsidethc window) and reiterate thc currentwindow size along with a current window edge as acknowledgment.’By the same token, the sender can, uponoccasion, choose to send more than a window’s worth of data on the possibility that the reccivcr will expand the window to accept it (of course, the sender must notsend more,than half the sequence number space at any time).Normally, we would expect the sender toabidebythc window limitation.Expansion of the window by the rcccivcr mcrcly allows more data to be ac- cepted. Vor the receiving HOST witha small amount of buffer space,astrategy of discardingallpacketswhose scqucnccnumbersdo not coincide with the currcnt left cdgc of the window is probably necessary, but itwill incur thc cxpcnsc of cxtra delay and overhead for retransmis- sion.
TCP INPUT/OUTPUT HAND,LING
The TCP has a component which handles input/output
(I/O) to and from the network4 When a packet has ar-
rived, it validatesthe addresses and places the packet onaqueue.A pool of buffers canbesetup t o handle arrivals, andif all available buffers areused up, succeeding arrivalscanbediscarded since unacknowledgedpacket will be retransmitted. On output,a smaller amount of buffering is needed, since process buffers can hold the data to be transmitted Perhaps double buffering mill be adequate. We make nc attemptto specify how the bufferingshould bedone except to require that it be able to service the network with as little overhead as possible. Packet sized buffers one or morering buffers, or anyothercombination art possible candidates. When a packet arrivesat thedestination TCP, itis placec on a queuewhich the TCP services frequently. For ex ample, the TCPcould be interrupted when a queue place ment occurs. The TCP then attempts toplace the packel textintothe properplace in’theappropriate proces!
receive buffer. If the packet terminates a segment, ther
it canbechecksummed and acknowledged.Placemeni may fail for several reasons.
I) Thedestination .process maynotbe. prepared t c
receive from the.etated source, or the destination port 11 may not exist. 2 ) There may be insufficient buffer space for the text 3) The beginningsequence number of thetext ma not coincide with the nextsequence number to bedeliverec to the process (e.g., the packet has arrived out of order) In the first case, the TCP shouldsimplydiscard thf packet(thusfar, noprovisionhasbeen made for err acknowledgments). Inthe second andthird cases, thc packet sequence numbercanbeinspectedto determinc whether the,packet textlies within the legitimate ivindow for reception. If it does, the TCP mayoptionally keep thc
packetqueued for later processing. If not,theTCI
can discard thepacket. I n either case theTCP car optionally acknowledgewith the current leftwindow edge
It may happen that the process receive buffer is no’
present in the active memoryof the HOST, but is stored or
secondary storage. If this is the case, the TCPcan promp
the scheduler to’bring in the appropriate buffer and thc packet can be queued for latcr processing.
If therc are no niore input buffers available to the TCI
for temporary queueing of ‘incomingpackets, and ifthc TCI’ cannotquicklyusethearriving data (c.g., a TCI to TCP message) , then thc packet is discarded. Assuminf a sensibly functioning system, no other processes than thc one for which the packet was intended should be affectec
by this discarding. If the delayed processing queue grow
Thiscomponent canserve tohandleotherprotocols whoss associated control programs are designated by internetwork destina tion address.
646 COMMUNICATIONS, ON IEEE TRANSACTIONS MAY 1974
. Current Message /
Sent. Acked Sent. Not Acked Not Sent Partial Next Message Current Ack Next Seq. No. iw,ndowNext’Read E n d i e a d 1 Next Write^ t
. Sire Transmit Buffet
Fig. 12. Transmit buffer layout.
ordering of delivered messages is also desired, both TCP’smustmaintain sufficient infornmtion to allow proper sequencing. When this information is also present at bothends, t,hen an association is said to have occurred. Note that we have not said anything about a path, nor anything which implies that eitherendbeaware of the condition of theother. Only when bothpartnersare prepared to communicate with’ each other has an associ- ation occurred, andit is possible that neitherpartner may be able to verify that anassociation exists until some data flows between them.
CONNECTION-FREE PROTOCOLS WITH ASSOCIATIONS Inthe ARPANET, the interface message processors (IMP’S) do not have to open and close connections from source to destination. The reasonfor this is that con- nectionsare, in effect,alwaysopen, since the addresk of everysource and destination is never5 reassigned. When the name and the place are static and unchanging, it is only necessary to label a packetwithsourceanddesti- nation to transmit it through the network.I n our parlance, every source and destination forms an association. I n thc casc of processes, however, we find thatport addresses are continually being used and reused. Some ever-present processes could be assigned fixed addresses
which do not change (e.g., the logger process). If we sup-
posed, however, that every T C P had an infinite supply of port addresses so that no old address would ever be reused, then any dynamically created port would be assigned the nextunusedaddress. I n such an environment,there could neverbe any confusion by source and destination T C P as to theintended recipient or implied source of each message, and all ports would bc associates. Unfortunately,,TCP’s (or moreproperly,operating systems) tend not to have an infinitesupply of internal port addresses.Thcse internal addresscs are reassigned aft‘er the demise of each port. Walden [ l Z ] suggests that a set of uniqueuniformexternal port addresses could be supplied by a ccntralrcgistry. A newly createdport could apply to the central registry for an address which the central registry would guarantee to beunused by any HOST system in thc network. Each TCY could maintain tablcsmatchingexternal names withinternal ones, and use the external ones for communicationwithother
HOST is connected to a different IMP.
5 Unless the IMP is physicallymoved toanothersite, or the
processes. This idea violates t.he premise that interprocess communica,tionshould not requirecentralizedcontrol. One would have to extend the central registry service to includeall HOST’S in all the interconnectednetworks to apply this idea to our situation, and we therefore do not att’empt to adopt it. Let us consider the situationfrom the standpointof the
TCP. In order to send or receive data for a given port,
the TCP needs to set up a TCB and RCB and initialize
the window size and left window edge for both. On thc receive side, thistaskmight even be delayed until the firstpacketdestined for a given port arrives. By con- vention, the firstpacketshouldbemarked so that tht receiver will synchronize to thereceived sequence number On the send side, the firstrequest totransmit coulc cause a TCBto be setupwith some initial sequenct number(say, zero) andan assumed window size. Thc receiving ‘I’CP canreject the packet if it wishes anc notify the sending TCP of the correct window size via thc acknowledgment mechanism, but only if either
- we insist that thefirst packet bea complete segment 2) an acknowledgment can be sent for the first packel (even if not a segment, as long as the acknowledg nlent specifies the next sequence number such t h a the source also understands that no bytes have beer accepted).
It is apparent, therefore, that thesynchronizing of windov
size and left window edge can be accomplished withou what would ordinarily’be called a connection setup. The firstpacket referencing a newly created RCE sent from ‘one associate to another can be marked with : bit which requests that the receiver synchronize his lef window edge with the sequencenumber of the arrivint
packet (see SYN bit in Fig. S). The TCPcan examine thc
source and destination port addresses in the packet an( intheRCBto decide whether to accept or ignore thc request.
Provision should be made for a destination process tc
specify that it is willing to LISTEN to a specific port o “any” port. This last idea permits processes such as thl logger process to accept data arrivingfrom unspecifiec sources. This is purely a HOST hatter, however. The initial packet may contain datawhich can be store( or discarded by the destination, depending on the avail ability of destination buffer spaceat thetime. In the othe direction, acknowledgment is returned for receipt of datr which also specifies the receiver’s window size. If the receiving T C P should wanttorejectthesyn chronization request, it merely transmits an acknowledg ment carrying a release (REL) bit (see Fig. 8 ) indicatini
that the destination port address is unknown or inacces
sible. The sending HOST waits for the acknowledgmen (after accepting or rejecting the synchronization request before sending the nestmessage or segment. This rejectiol is quite different from a negative data acknowledgment
We do not have explicit negative acknowledgments. If nc
acknowledgment is returned, the sending HOST ma:
CICRF AND K A H N : , PACKET 1\1<T\VORKINTICRCOMMUNICATION
retransmit without introducing confusion if, for example, the left window edgeis not changed onthe retransmission. I Because messages may be broken up into many packets ‘for transmission or during transmission, it will be neces- sary t o ignore the REL flag except in the case that the EM flag is also set’. This couldbeaccomplished either by t’heTCP or by the GATEWAY which could reset the flag
Ion all butthepacketcontainingtheset EM flag (see
Fig. 9). At the end of an association, the TCP sends a packet with ES, EM, and REL flags set.Thepacket sequence number scheme will alert the receiving TCP if there are .;till outstandingpacketsintransit which havenotyet arrived, so a prcmaturc dissociation cannot occur. To assure that both TCP’s are aware that the associ- ation has ended,wc insist that the rcceiving TCP respond
to the ItEL by sending a REL acknowledgment of its
own. Suppose now that a process sendsa single message to an associate including an RELalong with the data.Assuming
an RCB has beenprepared for the receiving TCPto
accept the data, the TCI’ will accumulate the incoming packets until the one marked ES, EM, BEL arrives, a t which point a REL is returned to the sender. The associ- ation is therebyterminatedandtheappropriateTCB and RCB are destroyed. If the first packet of a message
contains a SYN request bit and the last packet contains
~ES, EM, and REL bits, then datawill flow “one message at a time.” This mode is very similar to the scheme de- scribed byWalden [12], since eachsucceedingmessage can only be accepted at the receiver after a new LISTEN (likeWalden’s RECEIVE) command is issued bythe receiving process to itsserving TCP. Note thatonly if the acknowledgment is received by the sender can the associ-
ation be terminated properly. It hasbeenpointed out
thatthe receiver mayerroneouslyacceptduplicate transmissions if the sender does not receive the acknowl- edgment.Thismayhappen if thesendertransmitsa
duplicate message with theSYN and REL‘bits set and the
destinationhasalreadydestroyedanyrecord of the previous transmission. One wayof preventing this problem is todestroythe record of the association at the desti- nation only aftersome knownand suitablychosen timeout. However, this implies that a new association withthe same source and destination port identifiers could not be established until this timeout had expired. This problem
canoccureven with sequences of messages whose SYN
and REL bits are separated into different internetwork packets. We recognize that this problem must be solved, but do not go into further detail herc. Alternatively,both processes cansendonemessage, causing the respective TCP’s t o allocate RCB/TCB pairs at both ends which rendezvous with the exchanged data and then disappear. If the overhead of creating and dcstroyingRCB’sandTCB’s is small, suchaprotocol
S. Crocker of ARPA/IPT.
647
might be adequatefor most low-bandwidthuses. This idea might also form the basis for a relativelysecuretrans- mission system. If the communicating processes agree to change their external port addresses in some way known only to eachother(i.c.,pseudorandom),theneach message will.appear to the outside world asif it is part of a different association message stream. Even if the data is interceptedby a thirdparty,he will have no way of knowing that the datashould in fact be considered part of a sequence of messages. We have described the way in which processes develop associations with each other, thereby becoming associates for possible exchange of data. These associations need not involve the transmission of data prior to their formation and indeed two associates need not be able t o determine that they are associates until they attempt to communi- cate.
CONCLUSIONS
We have discussed some fundamental issues related to the interconnection of packetswitchingnetworks. I n particular, we have described a simple but very powerful and’ flexible protocolwhichprovides for variationin individualnetworkpacket sizes, transmission failures, sequencing, flow control, and the creation and destruction of process-to-process associations. Wehave considered some of the inlplementation issues that ariseandfound that the proposedprotocol is implementableby HOST’S of widely varying capacity. The next important step is to produce a detailed speci- fication of the protocol so that some initialexperinlents with it can be performed. These experiments are needed to determinesome of theoperationalparameters(e.g., how often and how far out of order do packets actually arrive;whatsort of delay is .there betweensegment acknowledgments;whatshouldberetransmissiontime- outs be?) of the proposed protocol.
ACI<NOWLEDGMENT
The authors wish t o thank a number of colleagues for helpful comments during earlydiscussions of international network protocols, especially R.Metcalfe,R.Scantle-
bury, D. Walden, and H. Zimmerman; D. Davies and L.
Pouzin who constructively commented on the fragmenta-
tionandaccounting issues; and S. Crocker whocom-
mented on the creation and destruction of associations.
REFERENCES L. Robertsand B. Wessler, “Computernetworkdevelopment to achieveresource sharing,”in 1970 SpringJointComputer Conf..AFIPSConf. Proc.. vol. 36. Montvale. N. J.: AFIPS Press: 1970, p p. ~545-549. L. Pouzin,“Presentationandmajor designaspects of the CYCLADEScomputernetwork,” in Proc. 3rd Data Com-
, - ~I ~
munications Symp., 1973. F. It. E. Dell, “Features of a proposed synchronous data net- work,” in Proc. 2nd Syrnp. Problems in the Optimization of Data Communications Systems, 1971, pp. 50-57.
