memory hierarchy For physically different kinds of memory there are significant differences in the time to read or write the contents of a particular location in memory, the amount of information that is read or written on a given occasion, the total volume of information that can be stored, and the unit costs of storing a given amount of information. To optimize its use and to achieve greater efficiency and economy, memory is organized in a hierarchy with the highest performance and in general the most expensive devices at the top, and with progressively lower performance and less costly devices in succeeding layers. The contents of a typical memory hierarchy, and the way in which data moves between adjacent layers, might be organized as follows. 1. Register – A single word held in each register of the processor; typically a word contains 4 bytes. This is sometimes not thought of as part of the hierarchy.2. Cache – Groups of words within the cache; typically a single group in the cache will hold 64 words (say 256 bytes), and there will be, say, 1024 such groups, giving a total cache of 256 Kbytes. Single words pass between the cache and registers within the processor. All transfers into and out of the cache are controlled entirely by hardware.3. Main memory – Words within the main (random-access) memory. On a very high performance system, groups of words corresponding to a group within the cache are transferred between the cache and the main memory in a single cycle of main memory. On lower-performance systems the size of the group of words in the cache is larger than the width of the memory bus, and the transfer takes the form of a sequence of memory cycles. The algorithm that controls this movement is implemented entirely in hardware. Main memory sizes are very variable – from as little as 1 Mbyte on a small system up to several Gbytes on a high-performance system.4. Online backing store – Blocks of words held on permanently connected backing store. There may be two somewhat distinct forms of activity here:(a) swapping device – pages (of say 4 Kbytes) or segments (up to many Mbytes) of memory held on a swapping device are transferred as complete units between their backing-store home and a page frame or segment area in main memory, under the control of an algorithm implemented by the software of the operating system but with hardware assistance to indicate when pages or segments are to be moved;(b) backing store – complete files, or clearly identifiable subsections of large files, are moved between the backing-store device and the main memory in response to explicit actions by the programmer, usually by a supervisor call to the operating system.5. Demountable storage – Complete files, backed up onto removable disks or magnetic tape within the file store system and the archiving system. Complete files are transferred in either direction. The creation of backup copies and the reinstatement of a backed-up file may be automatic, or may require direct intervention by the end user. For larger systems the backup medium is typically a modified form of a video or audio cassette system, possibly mounted in some form of computer-controlled cassette-handling robot system. Smaller systems may use a cassette system or floppy disks.

6. Read-only library – Complete files, and collections of associated files relating to a single application, held on read-only devices such as CD-ROM, or on a device with some form of write-protection control. Complete sets of files are read into the system from the read-only device, but for obvious reasons there are never any transfers from the system to the device.

The table indicates the typical access times, capacities, and unit costs for memory in each of these broad categories. Five main categories of system are considered:

E	enterprise servers
M	mid-range servers
W	workstations
D	desktop personal computers
L	laptop and fully portable personal computers

It is emphasized that this division is rather arbitrary, with no clear boundaries between systems. The capacity and performance of all systems continually increases, and there are many anomalies in the systems available in the marketplace, especially in the workstation, desktop, and laptop systems. The figures quoted are typical mid-range figures for performance, capacity, and costs (as at Spring 1995) and are clearly intended to be purely indicative of trends. With these caveats, the overall picture is reasonably consistent.

The following notes relate to the table: (1) In practice the registers on most current systems are part of a single chip, and it is difficult to identify accurately the cost of any single register.(2) On smaller systems the cache may be integrated with the processor proper, and again it is difficult to identify accurately the cost of any single register.(3) All systems use the same basic device for the actual storage. The higher unit costs on the larger systems arise from the inclusion of error-detection and error-recovery circuitry, and a wide data path between the memory and the cache.(4) The swapping device on large systems may take the form of semiconductor memory or high-performance disks, or a mix of the two.(5) The access time for any offline device is quoted as “minutes”. There are specific exceptions for offline tapes or cassettes held in some form of automated handler, where access time will typically be in the order of tens of seconds at worst. There is a similar exception for systems that handle multiple CD-ROMs. The unit cost for an online device includes the cost of the transport as well as the recording medium; for an offline device the cost quoted is purely for the recording medium.