Sneakernet is an informal term for the transfer of electronic information by physically moving media such as magnetic tape, floppy disks, compact discs, USB flash drives or external hard drives from one computer to another; rather than transmitting the information over a computer network.
Summary and backgroundEdit
Sneakernets, also known as trainnets or pigeonets, are in use throughout the computer world. Sneakernet may be used when computer networks are prohibitively expensive for the owner to maintain, in high-security environments where manual inspection (for re-classification of information) is necessary, where information needs to be shared between networks with different levels of security clearance, when data transfer is impractical due to bandwidth limitations, when a particular system is simply incompatible with the local network, unable to be connected, or when two systems are not on the same network at the same time. Because Sneakernets take advantage of physical media, security measures used for the transfer of sensitive information are respectively physical.
This form of data transfer is also used for peer-to-peer (or friend-to-friend) file sharing and has grown in popularity in metropolitan areas and college communities. The ease of this system has been facilitated by the availability of USB external hard drives, USB flash drives and portable music players.
The United States Postal Service offers a Media Mail service for compact discs, among other items. This provides a viable mode of transport for long distance Sneakernet use. In fact, when mailing media with sufficiently high data density such as high capacity hard drives, the throughput (data transferred per unit of time) as well as the cost per unit of data transferred may compete favorably with networked methods of data transfer.
This section does not cite any sources. (July 2018) (Learn how and when to remove this template message)
From an information theory standpoint, sneakernets can achieve tremendous throughput, but they suffer from high latency (see Network performance). The throughput of the network is directly proportional to the size of the transmitted file(s). Latency is based on the amount of time it takes to fully process the request for information. Latency would include the time it takes to write the storage media and the time to travel from point A to point B.
For example: Alice requests Bob to send her a DVD (4.7 GB) worth of information. Over the Internet, the latency for the file request may be milliseconds—Alice starts receiving the information nearly immediately—but at a modest broadband download speed of 50 kB/s it may take up to a day to complete the transfer. On the other hand, Bob could burn a DVD and deliver it to Alice in an hour. The latency was an hour, but the throughput of the transfer is roughly equal to a transfer rate of 1305 kB/s.
If an Airbus A380 were filled with microSD cards each holding 512 gigabytes of storage capacity, the theoretical total storage space onboard would be approximately 91 exabytes. A 4h47m flight from New York to Los Angeles would work out to a data transport rate of well over 5 petabytes per second, although this does not account for the time required to write to and read from the cards, which would almost certainly be much longer than the duration of the flight.
Similarly, as of 2016[update] the highest capacity backup tape format available is LTO-7, with a capacity of 6 TB. If a tape of this capacity were sent by overnight mail and were to arrive around 20 hours after it was sent, the effective data rate would be 664 Mbit/s. With networking technology, this magnitude of speed over this distance would be very difficult to attain without a costly dedicated connection as one would likely need to use several hops and have a connection that is not oversubscribed.
Sneakernets may also be used in tandem with computer network data transfer to increase data security. For example, a file or collection of files may be encrypted and sent over the Internet while the encryption key is printed and hand delivered or mailed. This method greatly reduces the possibility of an individual intercepting both the key and encrypted data.
Another way sneakernets are used together with network transfers is to provide an initial full backup as a base for incremental backups. In the case of a large (several terabyte) dataset that grows by just a few megabytes a day, the initial seeding of the data to be backed up would require an excessively long time upload over a network. One solution is to make a local copy of the data which is then physically relocated to the remote location and topped up with daily incremental backups over a network.
When Australia joined Usenet in 1983, it received articles via tapes sent from the United States to the University of Sydney, which disseminated data to dozens of other computers on the country's Unix network.
The May 2011 raid of Osama bin Laden's compound in Abbottabad, Pakistan revealed that he used a series of USB thumb drives to store his email drafts. A courier of his would then take the saved emails to a nearby Internet cafe and send them out to the desired recipients.
In September 2009, Durban company Unlimited IT reportedly pitted a messenger pigeon against South African ISP Telkom to transfer 4 GB of data 60 miles (97 km) from Howick to Durban. The pigeon, carrying the data on a memory stick, arrived in one hour eight minutes, with the data taking another hour to read from the memory stick. During the same two-hour period, only about 4.2% of the data had been transferred over the ADSL link. A similar experiment was conducted in England in September 2010; the "pigeonnet" also proved superior. In November 2009 the Australian comedy/current-affairs television program Hungry Beast repeated this experiment. The experiment had the team transfer a 700 MB file via three delivery methods to determine which was the fastest; A carrier pigeon with a microSD card, a car carrying a USB Stick, or a Telstra ADSL line. The data was to be transferred a distance of 132 km by road. The pigeon won the race with a time of approximately 1 hour 5 minutes, the car came in second at 2 hours 10 minutes, while the internet transfer did not finish, having dropped out a second time and not coming back.
Google has used a sneakernet to transport large datasets, such as the 120 TB of data from the Hubble Space Telescope. Users of Google Cloud can import their data into Google Cloud Storage through sneakernet.
The SETI@home project uses a sneakernet to overcome bandwidth limitations: data recorded by the radio telescope in Arecibo, Puerto Rico is stored on magnetic tapes which are then shipped to Berkeley, California for processing. In 2005, Jim Gray reported sending hard drives and even "metal boxes with processors" to transport large amounts of data by postal mail.
Wizzy Digital Courier provided Internet access to schools in South Africa with poor or no network connectivity by implementing UUCP on USB memory sticks. This allowed offline cached email transport and scoops of web pages to back-fill a web cache.
Data analytics teams in the financial services sector often use sneakernets to transfer sensitive corporate information and information obtained from data mining, such as ledger entries, customer data and financial statistics. There are several reasons for this: firstly, sneakernets can generally provide very high security (and possibly more importantly, they are perceived to be secure) due to the impossibility of a Man-in-the-middle attack or packet sniffing; secondly, the volumes of data concerned are often extremely high; and thirdly, setting up secure network links between the client business and the analytics team's facilities is often either impossible or an extremely convoluted process.
Very Long Baseline Interferometry performed using the Very Long Baseline Array ships hard drives to a data reduction site in Socorro, New Mexico. They refer to their data transfer mechanism as "HDOA" (Hard Drives On Airplane).
The Rigsum Sherig Collection project uses a sneakernet to distribute offline educational resources, including Kiwix and Khan Academy on a Stick, to hundreds of schools and other educational institutional in the Kingdom of Bhutan. Many of the schools in Bhutan have computers or IT labs, but no Internet connection (or a very slow one). The sneakernet, facilitated by teachers, distributes about 25 GB of free, open-source educational software to the schools, often using external hard disks.
In 2015 Amazon Web Services launched AWS Snowball, a 50 lb (23 kg), 50 TB device for transporting data to the AWS cloud; and in 2016 AWS Snowmobile, a truck to transport up to 100 PB of data in one load. For similar reasons, there is also a Google Transfer Appliance and an IBM Cloud Mass Data Migration device.
In 2017, the data of the first black hole image was recorded by the Event Horizon Telescope and had to be flown on hundreds of hard drives for processing. The image was released in April 2019. Each of the Event Horizon Telescope's Mark6 recorder receives digital data at a rate of 16 Gigabits/sec distributes it among a total of 32 hard disk drives grouped into 4 modules of 8 disks each. Over a typical five-day data collection campaign, each radio telescope collects 900TB of data. That data is then stored on from 1,000 to 2,000 of helium-filled 8TB hard drives, which amounts to about 7 petabytes (PB) of data. Petabytes of raw data from the telescopes were combined by highly specialised supercomputers hosted by the Max Planck Institute for Radio Astronomy and MIT Haystack Observatory.
Other alleged speakers included Tom Reidel, Warren Jackson, or Bob Sutterfield.
Although the station wagon transporting magnetic tapes is generally considered the canonical version, variants using trucks or Boeing 747s or C-5s and later storage technologies such as CD-ROMs, DVDs, Blu-Rays, or SD Cards have frequently appeared.
- The Terry Pratchett novel Going Postal (2004) includes a contest between a horse-drawn mail coach and the "Grand Trunk Clacks" (a semaphore line) to see which is faster to transmit the contents of a book to a remote destination.
- William Gibson's novel Spook Country (2007) also features sneakernets, with iPods being the storage device used to clandestinely move information.
The "valuable data file" has become a common MacGuffin in action films and television programs and indeed the motif of the "valuable letter or documents" (pre-electronic information storage technology) dates back hundreds of years.
- The film Johnny Mnemonic (1995), based on the short story by William Gibson, stars Keanu Reeves as a digital courier with 320 GB of corporate data transported in his head.
- The film Live Free or Die Hard (2007) depicts a digital thief attempting to download 500 TB of financial data to a suitcase-sized package.
- In the episode "Amen" of The Newsroom (2012), associate producer Maggie Jordan, after being told "it's 2011, we don't run film manually", is told that the video is taking too long to render on the local machine. It cannot feasibly be transferred over the network in raw form and must be run to the newsroom manually on a USB drive.
- In the film Elysium (2013), produced, written, and directed by Neill Blomkamp, Matt Damon as Max Da Costa downloads sensitive data into his brain implant in a data heist.
- In the film Snowden (2016), the titular character is seen evading security by carrying a data chip concealed in a Rubik's Cube.
- In the James Bond film Skyfall (2016), MI6 sets up a sneakernet due to security concerns.
- A large portion of the game Fallout:New Vegas involves a courier recovering and delivering a stolen casino chip containing part of a program for an anti missile defense system.
- "Oxford Dictionary". Retrieved 2016-09-09.
- Boutin, Paul (2002-08-26). "Sneakernet Redux: Walk Your Data". Wired News. Archived from the original on November 19, 2009. Retrieved 8 June 2010.
- Marquis, Bret (1983-03-29). "Australia joins USENET". Newsgroup: net.news.newsite. 467@sdchema.UUCP. Retrieved 14 February 2016.
- Apuzzo, Matt & Goldman, Adam (May 13, 2011). "How bin Laden emailed without being detected by US". The Washington Times. Associated Press. Retrieved June 29, 2012.CS1 maint: Uses authors parameter (link)
- McCullagh, Declan (May 13, 2011). "How bin Laden evaded the NSA: Sneakernet". Privacy Inc. CNET. Retrieved May 17, 2011.
- "SA Pigeon 'Faster than broadband'". BBC News. September 10, 2009.
- "BT feathers ruffled over pigeon-based file transfer caper". The Register. September 17, 2010.
- Pigeon flies past broadband in data speed race, BBC News Technology, September 16, 2010
- "The Great Australian Internet Challenge". ABC Television/Hungry Beast. 10 November 2009.
- "Google helps terabyte data swaps". BBC News. March 7, 2007. Retrieved May 23, 2010.
- Farivar, Cyrus (20 March 2007). "Google's Next-Gen of Sneakernet". Wired. Retrieved 5 February 2013.
- "Offline Media Import / Export". Google. Retrieved 29 January 2016.
- "A Conversation with Jim Gray". ACM Queue. 1 (4). July 31, 2003. Archived from the original on December 5, 2009.
Who would ever, in this time of the greatest interconnectivity in human history, go back to shipping bytes around via snail mail as a preferred means of data transfer?
- "Seeking Riches From the Poor", Megan Lindow, 04.23.04, Wired.com
- Rigsum Sherig Collection
- Khan Academy on a Stick
- "Only a Third of Government Schools Have Internet Access". Kuensel. April 18, 2013. Archived from the original on June 15, 2013.
- Fighting The State, Without The Web: North Korea's Sneakernet Insurgency
- How One Man Wants to Free North Korea With USB Drives and Pirated Movies
- North Korea’s Secret Movie Bootleggers: How Western Films Make It Into the Hermit Kingdom
- Balloon activist sends 'thousands of copies' of The Interview to North Korea
- In Cuba, An Underground Network Armed With USB Drives Does The Work Of Google And YouTube
- Kastrenakes, Jacob (October 7, 2015). "Amazon made a huge plastic box called Snowball so people can ship data to the cloud". The Verge. Retrieved October 8, 2015.
- Dignan, Larry (November 30, 2016). "AWS' Snowmobile data transport truck highlights why cloud giant is so damn disruptive".
- Sharwood, Simon (19 September 2017). "IBM packs 120TB into a carry-on bag, for snow-balling cloud uploads". The Register. Retrieved 19 September 2017.
- Ghosh, Pallab (April 10, 2019). "First ever black hole image released".
- Doeleman, Shep (April 2019). "Focus on the First Event Horizon Telescope Results".
- "Enhancing the Sensitivity of the EHT".
- Mearian, Lucas (August 18, 2015). "Massive telescope array aims for black hole, gets gusher of data".
- "Astronomers Capture First Image of a Black Hole".
- "Updated Textbook Explores Theoretical Basis of Networks". InfoWorld. 6 Feb 1989. Retrieved 16 April 2019.
- Poole, Steven (August 18, 2007). "Sign language". The Guardian. London: Guardian Media Group. Retrieved September 24, 2009.
- Ben Hui (March 1, 2006). "Haggle". University of Cambridge Computer Laboratory. Archived from the original on June 28, 2015.