The HIPPI Protocol

Jim Bell (


The High-Performance Peripheral Interface (HIPPI) protocol was designed to facilitate high-speed communications between very high-performance computers (such as supercomputers), and thereby to attempt to meet their I/O requirements.

This paper describes the HIPPI protocol in some depth, then surveys a few topics of advancement and extension of HIPPI.

Table Of Contents

List Of Figures

Overview of HIPPI

HIPPI is a very high-speed data transfer protocol, with the following properties, features, and limitations: The HIPPI protocol is defined, at several layers, by a collection of standards. The organization of these standards are shown below (from [1], [2]).


The components of the HIPPI protocol are described below.

HIPPI-PH (Physical Layer)

The HIPPI-PH standard defines the mechanical, electrical, and signaling of the HIPPI physical layer.

Note that since HIPPI is a simplex protocol, a full-duplex link would be achieved by another HIPPI connection in the opposite direction.

HIPPI's physical layer consists of a set of 50-twisted-pair copper cables (maximum length of 25 meters). Two options exist, based the desired data-rate: 800 or 1600 Mb/s. Since the latter is a logical extension of the former, this section will describe the former (800 Mb/s), then describe differences for the 1600 Mb/s channel.

Conceptual Transmission Elements

To understand signaling, one must first understand how HIPPI transmits data. The conceptual elements of data transmission are the connection, the packet, the burst, and the word.

Signal Lines

The HIPPI signals and timing are described below. Each line is dedicated to its own function. (Since HIPPI is simplex, this includes the data bus and parity bus.) The item in parentheses is the line direction: sourceis denoted by S , and destinationby D . (Note that the source initiates the data connection. One may think of a common example of a computer talking to an external disk drive where the computer is the data source and the disk is the destination.)


Though the HIPPI connection is considered asynchronous[4] (probably because connections can be requested asynchronously), it uses a 25 MHz (+/- 5%) clock, provided by the source, to transfer data synchronously within each burst. The clock signal is symmetrical (i.e., high and low pulses split the clock cycle evenly). During data transmission, one word is transmitted each clock cycle.

The CONNECT and READY signals are raised and dropped for a minimum of four clock cycles. Maximum signal times for most signals are not specified, to give both source and destination as much time as each needs during the times when the other is waiting on it.

The BURST and PACKET signals should be dropped for a minimum of two clock cycles, but the standard specifies that the destination should operate properly if BURST or PACKET are dropped for only one clock. This gives intermediate switches one clock cycle to synchronize.

Readers are referred to [2] for detailed timing descriptions.

1600 Mb/s Operation

For 1600 Mb/s operation, two identical cables are used instead of one. The doubled data-rate is achieved by doubling the width of the data-bus (64 bits instead of 32). The number of parity lines is also doubled (8 instead of 4), as is the number of interconnect lines in both directions (2 each, instead of 1). (This allows the hardware to make sure both cables are connected when required.)

HIPPI-FP (Framing Protocol)

This standard describes the format and content (including header) of each packet of user information. Note that other layers may be implemented above this one. Note also that this is the layer which splits higher layer packets to the 1 or 2 K-byte packets required by the physical layer.

The contents of a HIPPI-FP frame are as follows:

HIPPI-SC (Switch Control)

The most glaring limitation of HIPPI-PH is that it supports only a single point-to-point connection. Though this is necessary for practical purposes (to achieve the required data-rates), it goes against the dominant paradigm of networking: to allow many computers to share data. HIPPI-SC was developed as one workable solution to this quandary. It allows for a switching mechanism to be built which could allow multiple simultaneous point-to-point connections to occur (functionally similar to the way the phone company switches its point-to-point connections [i.e., phone calls][2]). Note that HIPPI-SC does notspecify any switching hardware or technology, but only provides functional mechanisms for these things to be used.

HIPPI-SC is closely coupled with HIPPI-PH. As described above, when the HIPPI-PH source raises its REQUEST line, it also specifies an I-Field on the DATA BUS . HIPPI-SC documents the format of the I-Field , as follows[2]:

It should be noted that although HIPPI-SC does help alleviate the the number of connections limitation, it doesn't provide the final solution. The number of switches required for interconnecting devices grows by the square of the number of devices: the cost of such a device also grows in proportion. As well, the size of HIPPI's 50-pair cable is large enough to cause switch-banks to grow at least linearly with the number of connections, and all connected computers still have to be within the 25m distance limitation. ( Serial HIPPIis providing better solutions in this area.)

Mapping HIPPI to Other Protocols

Three other standards for HIPPI exist, which are built upon the HIPPI-FP layer. These are: With the overwhelming popularity of the TCP/IP and related protocols, all built upon the LLC layer (described by IEEE 802.2), the HIPPI-LE standard is, of the above, by far the most important. The following lists the most important mappings of HIPPI-LE:

Note that the above mappings into the D1_Area (and, possibly, the D2_Area ) are only one possible interpretation of these HIPPI-FP areas. Therefore, a virtually unlimited number of other mappings could be built upon the HIPPI-FP layer, and the HIPPI-FP layer (and the HIPPI-PH layer) would be completely independent of each of these.

HIPPI's Origin

The HIPPI protocol standard suite was designed by ANSI Task Group X3T9.3, which was initiated and chaired by Don Tolmie of Los Alamos National Laboratory. Tolmie says that the project was first proposed in January, 1987, with products available by late 1988[1] (but van Praag, et. al.[6] date its beginning at 1989, and note that the standard is dated 1991).

HIPPI was designed to be a "fire hose" for moving data, as well as a very implementable standard. The following design guidelines were used[1]:

Some important design decisions were made based on the above guidelines: Franta and Hughes[7] consider HIPPI a "standardized variant of Cray's proprietary HSX channel, which [also] operates at 800 Mb/s." This indicates that Cray had a significant influence on the standard, though I did not find that elsewhere in the literature.

Current Topics in HIPPI

This section reports some recent and current topics in HIPPI. Notably, most of these topics are attempts to overcome one of the two inherent limitations of HIPPI: the maximum distance between nodes and the number of connections.

Serial HIPPI

Serial HIPPI is an attempt to overcome the 25m distance limitation by the use of &HIPPI Modems& of sorts: devices that connect to HIPPI-PH via the standard 50-pair connection on one side, but which serialize the data and use another technology to transmit it longer distances to another complementary device, which converts it back to HIPPI-PH for transmission to the HIPPI destination.

Two flavors of such devices are discussed[2,3]:

These devices, though transparent to HIPPI-PH, inevitably increase the turn-around time of its signals. But due to the design of HIPPI-PH, latencies are well hidden, and only show up in making and breaking connections, which is typically reasonably infrequently.

High Speed SONET Extensions

Hughes and Franta[3] propose a HIPPI extender device which will use SONET's STS-12c technology to transport HIPPI packets. They propose this device as a lower-cost alternative to other approaches using multiple slower-speed long-distance paths (OC3c).

Much like Serial HIPPI, they create a set of "HIPPI terminating devices" which serialize the data, then place it into an STS-12c payload for transmission. The down-link device then converts it back to HIPPI for transmission into the destination HIPPI link. They lay out a physical layer convergence protocol (PLCP) which maps HIPPI bursts into STS-12c rows.

Nowhere do they attempt to match HIPPI's 800 Mb/s data rate. STS-12c (over a OC-12 optical carrier) has a maximum data-rate of 622.08 Mb/s, and with their PLCP, the effective data-rate becomes 589.824 Mb/s. They do not directly address this discrepancy, but mention one customer benchmark in which supercomputers would sustain a 400 Mb/s data-rate, with or without their device in the data path. Apparently, the assumption is that steady-state data-rates of 800 Mb/s will not occur.

Their devices rely heavily on large RAM stores in the down-link extender (i.e., on the SONET receiving end). Their rough estimate is that as much as 2 x band_width x delay_bytes storage is required in the down-link extender. For a 3,000-mile link, they estimate that over 1 Megabytes of data may be transmitted by the up-link extender before the HIPPI source can be stopped. This data must be buffered in the down-link extender.

The extenders they designed had the following features:

HIPPI Connection Management

As mentioned above, the HIPPI standards do not provide a complete, general-purpose interconnection solution. Therefore, work is being done to investigate methods for providing such a solution (without sacrificing HIPPI's data transfer speed or efficiency).

Chlamtac, Ganz and Kienzle[4] analyze the theoretical and simulated performance of various connection management policies . A connection management policy is a method by which connections (a network resource) are shared. Three such policies are considered:

  1. Centralized-- One centralized, distinct management system, closely integrated with the switches, maintains all connection management information for the entire network. When a connection is established or broken, this system performs the next assignments which need to be made.
  2. Broadcast-- A separate bus-type communications system allows nodes to broadcast information regarding the HIPPI links, and therefore manage connections by policies built upon this shared knowledge.
  3. Distributed-- Each HIPPI node works with no explicit knowledge of any others. Management policies are then built upon random waiting and retrying methods.
They provide a detailed statistical analysis of each method (which is beyond the scope of this paper). They come to the following conclusions:

Real-World Uses of HIPPI

CERN, the European high energy physics laboratory has several HIPPI applications well underway[6].

Though they mention this in passing, it's interesting to note some of the standard HIPPI components available (as of 1992):

They are using the HIPPI VMEbus interface to build a VMEbus interface data collection unit, which will be used in the Large Hadron Collider among other things.

Annotated Bibliography

  1. D. Tolmie and J. Renwick, "HIPPI: Simplicity Yields Success", IEEE Network, Vol. 7 No. 1, January 1993, p 28-32. Describes basic operation of HIPPI, and origin of standards. Mentions goals of the standards committee.
  2. D. Tolmie, "High-Performance Parallel Interface (HIPPI)", (from "High Performance Networks, Technology and Protocols," edited by Tantawy), Kluwer Academic Publishers, 1994. Describes operation of HIPPI in detail, at several levels. Describes packet formats in detail.
  3. J. P. Hughes, W. R. Franta, "Geographic extension of HIPPI channels via high speed SONET", IEEE Network, Vol. 8 No. 3, May-June 1994. p 42-53. Describes a method for using SONET to carry HIPPI bursts. Also mentions the current state of Serial HIPPI.
  4. I. Chlamtac, A. Ganz, M. G. Kienzle, "An HIPPI Interconnection System", IEEE Transactions on Computers, Vol. 42 No. 2, Feb 1993. p 138-150. Describes a proposed HIPPI interconnection system using a cross-bar switch network. Performs a reasonably extensive mathematical model of the performance of such a network.
  5. I. Chlamtac, M. G. Kienzle, "Multitasking in high-speed interconnection systems.", Computer Networks and ISDN Systems, Vol. 25 No. 6, Jan 1993, p 701-716. Overlaps with [4], and also analyzes access disciplines (i.e., methods by which processes compete or share a single resource, such as a HIPPI port, which can only be connected to one destination at a time).
  6. van Praag, A.; Anguelov, T.; Burckhart, D.; McLaren, R. A.; van der Bij, H. C.; Bovier, J.; Cristin, P.; Haben, M.; Jovanovic, P.; Kenyon, I.; Staley, R.; Cunningham, D.; Watson, G.; Green, B.; Strong, J., "HIPPI developments for CERN experiments.", IEEE Transactions on Nuclear Science, Vol. 39 No. 4, Aug 1992. p 880-885. Describes CERN's work with HIPPI devices. Describes a HIPPI VMEbus interface computer they are designing (from standard components).
  7. W. R. Franta and J. P. Hughes, "Extended High Speed Networks Employing HIPPI Switches, High Speed WANs, and FDDI Rings", Journal of High Speed Networks, Vol. 2, 1992. p 167-192. This is an overview of many Wide Area Network technologies, with one brief overview section on HIPPI. Not directly on topic, this article makes several observations of networking in general, and use these to attempt to predict the future of networking. They consider HIPPI to be a significant part of the future of networking.
  8. K. Hung-Chang, A. Nilsson, D. Winkelstein, L. Bottomley, "Traffic Measurements on HIPPI Links in a Super-computing Environment", (from "Asynchronous Transfer Mode Networks", edited by Y. Viniotis and R. O. Onvural), Plenum Press, New York, 1993. Heavy duty mathematical analysis of HIPPI performance.

The HIPPI Standards:

Related Standards & RFC's


The High-Performance Peripheral Interface. This protocol, used to transfer data at extremely high rates, is the focus of this paper.
Physical Layer.
Framing Protocol.
Link Encapsulation.
Intelligent Peripheral Interface. A set of disk and tape commands specified by the standards: ANSI/ISO 9318-3:1990 and ANSI/ISO 9318-4:1990.
Megabits per second.
Mega-hertz, 10^6 cycles per second.
The data transmitting node (device, computer, etc.).
The data receiving node (device, computer, etc.).
Longitudinal Length Redundancy Check-word. This word, an even longitudinal parity word transmitted (by the physical layer) at the end of each burst, allows error detection by higher layers. All odd-bit and 2-bit errors, and 99.99% of 4-bit errors are detected.
Least Significant Bit.
Most Significant Bit.
Protocol Data Unit. Defined by the IEEE 802.2 standard.
Synchronous Optical NETwork. A high-speed networking service provided by phone service providers.

Other Reports on Recent Advances in Networking 1995
Back to Raj Jain's Home Page