 | ||
Sneakernet is an informal term describing the transfer of electronic information by physically moving media such as magnetic tape, floppy disks, compact discs, USB flash drives or external hard drives from one computer to another; rather than transmitting the information over a computer network.
Contents
The term, a tongue-in-cheek play on net(work) as in Internet or Ethernet, refers to the wearing of sneakers as the transport mechanism for the data.
Summary and background
Sneakernets, also known as trainnets or pigeonets, are in use throughout the computer world. Sneakernet may be used when computer networks are prohibitively expensive for the owner to maintain, in high-security environments where manual inspection (for re-classification of information) is necessary, where information needs to be shared between networks with different levels of security clearance, when data transfer is impractical due to bandwidth limitations, when a guest laptop is incompatible with the local network, or simply when two computers are not powered up at the same time or lack the correct interconnecting cabling. Because sneakernets take advantage of physical media, different security measures must be taken into account for the transfer of sensitive information.
This form of data transfer is also used for peer-to-peer (or friend-to-friend) file sharing and has grown in popularity in metropolitan areas and college communities. The ease of this system has been facilitated by the availability of USB external hard drives, USB flash drives and portable music players.
The United States Postal Service also offers a Media Mail service for compact discs, among other items. This provides a viable mode of transport for long distance sneakernet use. In fact, when mailing a sufficiently large hard drive or a spindle of DVDs, the throughput (amount of data per unit time) may compete favorably with other methods of data transfer.
Theory
From an information theory standpoint, sneakernets can achieve tremendous throughput, but they suffer from high latency (see Network performance). The throughput of the network is directly proportional to the size of the transmitted file(s). Latency is based on the amount of time it takes to fully process the request for information. Latency would include the time it takes to write the storage media and the time to travel from point A to point B.
For example: Alice requests Bob to send her a DVD (4.7 GB) worth of information. Over the Internet, the latency for the file request may be milliseconds—Alice starts receiving the information nearly immediately—but at a modest broadband download speed of 50 kB/s it may take up to a day to complete the transfer. On the other hand, Bob could burn a DVD and deliver it to Alice in an hour. The latency was an hour, but the throughput of the transfer is roughly equal to a transfer rate of 1305 kB/s.
The theoretical capacity of an Airbus A380 filled with microSD cards is 91,000,000 Terabytes, resulting in a 5.284 PB/s flight from New York to Los Angeles, if time to write and read the cards is not considered.
Similarly, as of 2016 the highest capacity backup tape format available is LTO-7, with a capacity of 6 TB. If a tape of this capacity were sent by overnight mail and were to arrive around 20 hours after it was sent, the effective data rate would be 664 Mbit/s. With networking technology, this magnitude of speed over this distance would be very difficult to attain without a costly dedicated connection as one would likely need to use several hops and have a connection that is not oversubscribed.
Sneakernets may also be used in tandem with computer network data transfer to increase data security. For example, a file or collection of files may be encrypted and sent over the Internet while the encryption key is printed and hand delivered or mailed. This method greatly reduces the possibility of an individual intercepting both the key and encrypted data.
Another way sneakernets are used together with network transfers is to provide an initial full backup as a base for incremental backups. In the case of a large (several terabyte) dataset that grows by just a few megabytes a day, the initial seeding of the data to be backed up would require an excessively long time upload over a network. One solution is to make a local copy of the data which is then physically relocated to the remote location and topped up with daily incremental backups over a network.
Usage examples
When Australia joined Usenet in 1983, it received articles via tapes sent from the United States to the University of Sydney, which disseminated data to dozens of other computers on the country's Unix network.
The May 2011 raid of Osama Bin Laden's compound revealed that he used a series of USB thumb drives to store his email drafts. A courier of his would then take the saved emails to a nearby Internet cafe and send them out to the desired recipients.
In September 2009, Durban company Unlimited IT reportedly pitted a messenger pigeon against South African ISP Telkom to transfer 4 GB of data 60 miles (97 km) from Howick to Durban. The pigeon, carrying the data on a memory stick, arrived in one hour eight minutes, with the data taking another hour to read from the memory stick. During the same two-hour period, only about 4.2% of the data had been transferred over the ADSL link. A similar experiment was conducted in England in September 2010; the "pigeonnet" also proved superior. In November 2009 the Australian comedy/current-affairs television program Hungry Beast repeated this experiment. The experiment had the team transfer a 700 MB file via three delivery methods to determine which was the fastest; A carrier pigeon with a microSD card, a car carrying a USB Stick, or a Telstra ADSL line. The data was to be transferred a distance of 132 km by road. The pigeon won the race with a time of approximately 1 hour 5 minutes, the car came in second at 2 hours 10 minutes, while the internet transfer did not finish, having dropped out a second time and not coming back.
Google has used a sneakernet to transport large datasets, such as the 120 TB of data from the Hubble Space Telescope. Users of Google Cloud can import their data into Google Cloud Storage through sneakernet.
The SETI@home project uses a sneakernet to overcome bandwidth limitations: data recorded by the radio telescope in Arecibo, Puerto Rico is stored on magnetic tapes which are then shipped to Berkeley, California for processing. In 2005, Jim Gray reported sending hard drives and even "metal boxes with processors" to transport large amounts of data by postal mail.
Many film editing and visual effects companies transfer large film-scans using hard drives shipped via courier (to reduce bandwidth bills, and to reserve bandwidth for more time-critical transfers).
Wizzy Digital Courier provided Internet access to schools in South Africa with poor or no network connectivity by implementing UUCP on USB memory sticks. This allowed offline cached email transport and scoops of web pages to back-fill a web cache.
When home broadband access was less common, many people downloaded large files over their workplace networks and took them home by sneakernet. Today, as home broadband is more common, sometimes technical workers at institutions with congested WAN links do the reverse: downloading data at home in the evening and carrying the files to work on USB flash drives.
In the early demoscene, the primary method of exchanging data was using snail mail to exchange floppies between groups. Each group usually had at least one person designated as a swapper, who would exchange news, data and productions with swappers from other groups this way. The best swappers were known to send and receive over 100 mails a month.
Online DVD rental services such as GameFly, or those formerly provided by Netflix, are effectively sneakernets. They deliver data, stored on DVDs and occasionally other forms of digital media, via regular postal mail.
Petroleum seismic surveys routinely record field data which are many terabytes in size. A cluster computer is required to process these data, and may take a year or more, during which time the field crew will wish to work in other areas. The field data are generally hand-carried on tape, and increasingly on hard disk inserts, to the processing centre.
Data analytics teams in the financial services sector often use sneakernets to transfer sensitive corporate information and information obtained from data mining, such as ledger entries, customer data, and financial statistics. There are several reasons for this: firstly, sneakernets can generally provide very high security (and possibly more importantly, they are perceived to be secure) due to the impossibility of a Man-in-the-middle attack or packet sniffing; secondly, the volumes of data concerned are often extremely high; and thirdly, setting up secure network links between the client business and the analytics team's facilities is often either impossible or an extremely convoluted process.
Very Long Baseline Interferometry performed using the Very Long Baseline Array ships hard drives to a data reduction site in Socorro, New Mexico. They refer to their data transfer mechanism as "HDOA" (Hard Drives On Airplane).
The Rigsum Sherig Collection project uses a sneakernet to distribute offline educational resources, including Kiwix and Khan Academy on a Stick, to hundreds of schools and other educational institutional in the Kingdom of Bhutan. Many of the schools in Bhutan have computers or IT labs, but no Internet connection (or a very slow one). The sneakernet, facilitated by teachers, distributes about 25 GB of free, open-source educational software to the schools, often using external hard disks.
North Korean dissidents have been known to smuggle flash drive filled with western movies and television shows, largely in an effort to inspire a cultural revolution.
El Paquete Semanal is a roughly 1TB compilation of media, distributed weekly throughout Cuba via portable hard drives and USB memory sticks.
In 2015 Amazon Web Services launched AWS Snowball, a 50 lb (23 kg), 50 TB device for transporting data to the AWS cloud; and in 2016 AWS Snowmobile, a truck to transport up to 100 PB of data in one load.
Non-fiction
Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
—Tanenbaum, Andrew S. (1989). Computer Networks. New Jersey: Prentice-Hall. p. 57. ISBN 0-13-166836-6.
The original version of this quotation came much earlier; the very first problem in Tanenbaum's 1981 textbook Computer Networks asks the student to calculate the throughput of a St. Bernard carrying floppy disks (in 1981 the standard 8 inch floppy held 241 kilobytes of data per side, though various formatting and data blocking techniques could get that as high as 800 kilobytes per side, while the 5.25 inch floppy held 180 kilobytes per side). The first USENET citation is July 16, 1985, and it was widely considered an old joke already.
There's a lot of band-width in a station wagon.
-Gruenberger, Fred (1971). Computing: A Second Course. San Francisco: Canfield Press. p. 138. ISBN 978-0063834057. Retrieved 24 January 2017.
Other alleged speakers included Tom Reidel, Warren Jackson, or Bob Sutterfield. The station wagon transporting magnetic tapes is the canonical version, but variants using trucks or Boeing 747s or C-5s and later storage technologies such as CD-ROMs, DVDs, Blu-Rays, or SD Cards have frequently appeared.
Fiction
The Terry Pratchett novel Going Postal includes a contest between a horse-drawn mail coach and the "Grand Trunk Clacks" (a semaphore line) to see which is faster to transmit the contents of a book to a remote destination.
The "valuable data file" has become a common MacGuffin in action films and television programs, and indeed the motif of the "valuable letter or documents" (pre-electronic information storage technology) dates back hundreds of years.