CyberAll: A Personal Store for
Everything[1]
Gordon Bell
10 July 2000
Technical Report
Microsoft Research
Microsoft Corporation
One Microsoft Way
Redmond, WA 98052
CyberAll: A Personal Store for Everything
Gordon
Bell, Senior Researcher
Microsoft Bay Area Research Center
San Francisco, CA
CyberAll is a project to
encode, store, and be able to retrieve all of a person’s information for
personal and professional use. The archive includes books, CDs, correspondence
(i.e. letters, memos, and email), transactions, papers, photos and albums, and
video.
In 2000, only 16 gigabytes are
required to store all the media in my personal and professional life -- costing
$160 of disk storage. Two gigabytes are expected to be added next year. Encoding, indexing, and data-management
costs swamp the storage cost. The challenge is to automate capture, search, and
retrieval.
One challenge is to automate
the capture, search, and retrieval so that it comes close to the storage
cost. It is inconceivable to think of
manually managing or purging this electronic file since the storage costs are
only $100. Indeed, copies are stored in
2 or 3 locations for redundancy.
A second challenge is adding
facilities that justify the cyberization investment. Use includes everything
from retrieval of professional information to personal and family apps such as
“playing” CDs, photos, and video on home computers and TV sets.
CyberAll is a project
to encode, store, and be able to retrieve all of a person’s information for
personal and professional use. The archive includes books, CDs, correspondence
(i.e. letters, memos, and email), transactions, papers, photos and albums, and
video.
In 2000, only 16 gigabytes are required to store
all the media in my personal and professional life. This costing $160 of disk
storage. Two gigabytes are expected to be added next year. Encoding, indexing, and data-management
costs swamp the storage cost. The challenge is to automate capture, search, and
retrieval.
CyberAll is a personal ontology (Huhns and Stephens, 1999) in contrast to a library (Lesk, 1997) or Kahle’s effort to archive the web and Television channels www.archive.org. It is my store for documents, photos (e.g. people, computing artifacts), music, and videos as described by Bush (1947) and Gates (1997).
CyberAll also holds reference articles e.g. Amdahl’s Law, clipped graphs that heretofore would be physically stored, manuals e.g. Digital PDP-1[2], CDC 6600, and magazines. At present, books are in “atomic” form; but CyberAll will include them as they become e-books. Http://www.research.microsoft.com/~gbell holds co-authored books.
Within the next decade personal computers will each store a terabyte. In 2000, 40 gigabyte drives costing $400 are more than adequate to hold the content for most of a professional’s lifetime reading, presentations, and audio recordings. A CD encoded at 128 kilobits per second can be stored at a cost of $0.60. A user’s CD collection is likely to use more storage space than her computer generated and scanned paper files.
The next phase of CyberAll will capture conversations, interviews, meetings, and presentations. Recording speech from one’s personal and professional lives will require over a terabyte (8 kilobits per second) – but only a modest 25GB/year .
Video is even more challenging. For home use, a terabyte holds 500 hours of DVD quality video and 1500 CDs, but more compression increases the capacity by a factor of at least 10. Recording a lifetime of everything seen requires 100 terabytes. Doing this economically is still more than a decade away – now it would cost more than $10,000 per year. But in two decades, it should cost only $100 per year and require an infrastructure unlike anything we currently know.
The technologies for cyberization are improving at the rate of Moore’s Law -- doubling every 18 months. These include processor speed, storage capacity, scanner speed and accuracy, camera resolution and software, OCR accuracy and capability (e.g. scan to HTML), audio and video encoding, printing and display, and standards. Thus, one can always wait for a better system or standards – things will be twice as good in 18 months. However, content and capture cost are almost acceptable, and the longer we wait, the more information is lost forever– so it is important to start.
Based on my experience of being able to only go back 25 years for some content, the most serious concern of CyberAll is choosing formats that will be readable in 10 to 50 years. CyberAll requires a mechanism for carrying data forward from legacy media, systems, and programs.
The motivation for CyberAll ranges from the technical challenges i.e., “because we can” to a desire to have an exhaustive archive. Electronic filing cabinets such as Ricoh’s eCabinet (Ricoh, 1999) accept both computer generated and scanned documents and index the documents it holds! Filing systems e.g. Windows 2000, Office index their documents.
Many share a “pack rat” mentality to store everything to remind ourselves or others. CyberAll is an attic to store everything that can answer a question or explain what it was like when. It is a memory aid and a device to help tell stories. When its photos are displayed on a TV set or computer in a screen saver fashion, it provides a pleasant ambiance that jogs our memory. For some, this might mean storing everything e.g. our first drawings, grade cards, and home videos. New web sites e.g. www.123456789.net, www.legacy.com , www.forevernetwork.com and www.memorymountain.com offer to store letters, essays, photos, videos and stories “forever” to and pass them to their future generations.
CyberAll is also project to understand the problems of coping with the exponential increase in the amount of information (e.g. email, web pages, pictures, audio, and video) that is becoming part of both our personal and professional lives. Given the tools to mass produce documents, we are forced to become filing clerks!
The goal of CyberAll is both to encode everything and to eliminate paper that is used for storage (filing) and transmission. Paper will remain a dominant reading interface where its advantages are well known. Many documents that represent money i.e. plain old money, notes, stock, and cancelled checks have to be retained[3].
Table 1 shows the kinds of content that occur in an individual’s personal and professional lives for archival (mainly reference) and daily (working) use, e.g. contracts, email, and music. This includes encoded legacy content e.g. papers, photos, audio and video tapes to computer created papers, presentations, photos, “ripped” CDs, and video tapes. CyberAll can play all of the content from photos to CDs on computers, home stereo, and TV sets.
|
Table 1. Data-types and use for timeliness and user
context. |
||
|
User Context / Timeliness |
Personal (entertainment and
personal finance related) |
Professional (work related) |
|
Archival (historical reference) |
Documents, photos and
photo albums, music, video memory-aid, entertainment, medical
history, progeny |
Books, papers, reference
documents |
|
Working |
Documents, email, photos,
audio including CDs, video communication, entertainment, financial records |
Documents, email |
CyberAll is for personal use as opposed to providing a general server. CyberAll operates in my COMOHO (commercial office, mobile office, and home office) environment, providing access anywhere, anytime. The main desktop computer in the BARC lab (CO) holds all files and is well backed up.
The author’s portable computer (MO) contains a large subset “cache” of the CO. It is the principle computer, used in the MOHO environments. In MO locations, modems, hotel LANs, etc. communicate via the corporate network to CO for “uncached” documents.
In the HO, ADSL and cable modems link to CO, allowing audio and picture files to be “played”.
By keeping all information, CyberAll should be able to provide a useful set of answers and services including:
· Recall a Chicago hotel stay over the last ten years or a restaurant or wine from a dinner in Paris about four years ago.
· Find a cancelled check or receipt.
· Show figures from papers on supercomputers during 1980-1990.
· Find articles, papers, etc. that mention Amdahl’s laws, including the original articles.
· Recall email and letters to or about X about five years ago even though it was not specified to be a letter or email. List letters, recommendations, and papers written in 1989.
· Display an album from a fishing trip or taken during July 1999 on the TV set, or display all the photos randomly on a large flat panel display.
· Play a set of selections on a particular computer or the home stereo.
CyberAll is currently held in the Windows file system. A decision was made to not use a database. This was based on: variation of document types; cost to create and maintain database columns, keywords, or meta-data; inflexibility of moving or modifying files in an established database; concern that any database is not a “golden” data-type and hence is likely to become obsolete; a belief that programs should be able to automatically extract any relevant meta-data e.g. letters, forms; and the ability of ordinary indexing and searching to solve most personal needs.
Items are stored in a relatively flat 2- or 3-level folder hierarchy with a few dozen folders in the first level and an average of 4 folders in the second level. A plethora of specialized music database programs manage the encoding, organization, and playing of CDs, music files and music sources. Photos represent a challenge. The photo collection is called the shoebox, and indeed has that flavor. My database colleagues down the hall have yet to convince me that they can do better than “grep” searching the free text or viewing thumbnails.
The author has also used descriptive file names to aid retrieval. A name might include subject, organization, keywords and a date. Many file types e.g. Word, JPEG have extensive meta-data. JPEG photos include title, subject, location, description, category, keywords, dates (taken, modified, etc), and camera information.
Documents are retrieved by searching file contents. For example, searching is instantaneous using AltaVista or the Windows 2000 file system. Eventually all the information of or about a file inherent in the file is needed. Systems need to “understand” the documents e.g. letters, receipts they hold.
Photos are stored as individual photos in a set of personal and professional folders, and albums -- when there is a story. Retrieval is by date, photo name or any other text attributes when they have been so labeled. Most of us are unwilling to label and describe each photo since a year of photos by a prolific amateur could take several days to label. Thus, the alternative is viewing a folder of thumbnails and using emerging image searching programs.
A photo (or a pointer aka “shortcut” to it) is stored in every folder where a user might expect to find it. Folders provide an organized, yet open ended filing structure. Folders are grouped as: time-based events e.g. trip, party, conference; and subjects e.g. family member, hobby, mountain scene, food. One can easily have three attributes or folder sets where a single photo (or pointer) is stored e.g. French ‘97 trip, French mountain scenes, and all mountain scenes that include France. Sunsets might get a fourth filing. Each time a new, useful category is found, thumbnails are made and inserted in an appropriate folder. Tools for compound searches e.g. mountain and sunsets would be useful.
Obviously, there are a plethora of functions that can be invented to facilitate filing, labeling, and retrieval. Arcsoft’s Photobase creates albums with searchable keywords and audio segments for each image. Speech offers great potential to assist filing!
Legacy data-types, e.g. CD, paper, photo, and videotape have stood the test of time. Various tools allow them to be cyberized. In contrast, for computer created items, the application program that created an item may often no longer be available – so items are essentially lost. Over the long term, older versions of complex programs like databases, word processors, and computer games may no longer run on new systems. Only .txt format seems to be readable over decades!
Information must be held in a few golden primitive formats because these have to be supported forever! CyberAll documents are stored in at least two formats to increase the likelihood of reading the document in the future. Black and white documents are scanned and retained in tagged image format (TIF) and also converted to some OCR’d form e.g. PDF for retrieval. Some documents are converted to Word or HTML for searching, viewing, printing, and even editing. For example, a scanned copy of the 1889, 13-page Hollerith patent TIF file requires 700 Kbytes and 79 Mbytes for black and white and color, respectively. A PDF file of the image for limited on-screen viewing, printing, and searching is about 1 MByte. DjVu (1999) stored color documents appear to encode compound color and text documents in half the size of other formats.
JPEG, HTML, PDF, RTF/DOC and TIF are “golden” formats. PowerPoint is a container for photos.
An HP Digital Sender was used to scan to either black and white or color TIF or PDF[4]. For most working documents PaperPort is used to scan to a TIF dialect with implicit OCR. It is difficult, though necessary, to cut a relatively rare bound book, paper, or report apart to scan and discard. Some documents (e.g. engineering notebooks, notes) have not been scanned due to readability and contrast. If a document needs to be permanently preserved, it is converted to a golden format to increase its likelihood of permanency.
TIF format is the basis for virtually all OCR and page input programs:
Document ® Scan ® TIF future versions of TIF include OCR’d text
|® Acrobat ® PDF(with OCR’d text)
|® e.g. Omnipage +manual effort ® DOC | HTML & Simages
Various OCR programs can recognize and convert a document into a repurposed, near likeness of the original or even an HTML page. The MHT format, derived from MIME, can hold the collection of files for an HTML page in a single file. The evolution of TIF and HTML-XML to hold different image encodings, including recognized text, JPEG, and GIF objects will make scanning more convenient, economical and useful.
Future TIF standards include the image, OCR’d text for searching, and meta-data (e.g. various dates, author, and keywords that further describe the document). Scanners that directly connect to a personal computer usually just provide bitmap images and, depending on the interface software, images can be stored in a various formats.
Photos are scanned into folders. Albums hold stories e.g. a trip, birthday party, or a period of a family’s life. PowerPoint is the main container for albums, but in addition the photos are retained in folders since one photo may appear in several albums. PowerPoint can be converted directly to the html for web hosting, or alternatively an html document containing the photos can be created using various web authoring tools.
SPhotos ® Scan ® SJPEG ® PowerPoint ® .PPT
PDF albums are used to encode legacy paper albums
(multiple pictures mounted on a page).
All photos are JPEG. TIF is not used as the intermediate images format because of size. Kodak’s photo CD conversion service, and Nikon and HP scanners were used for photo input.
As a rule, simple items such as a page, photo, or slide costs about $1 from commercial services. Ten page articles can be scanned directly into PDF in about two minutes with the HP Sender captured in Acrobat format at 3 pages per minute using a 400 MHz PC. Photo scanners take 20 seconds to 2 minutes per photo.
One may want to recognize a document and convert it to a perfect, editable document e.g. DOC/RTF, HTML. This requires “perfect” recognition together with the need to format the document exactly like the original. Such a document is being re-published. To scan, recognize, and edit a page can easily require 10 minutes to create a formatted document that is suitable for repurposed use.
The time to encode or “rip” a CD depends on the CD reader speed, tools and availability of databases that can be used to create labels. CDs took roughly 10 minutes of attention time to read and label the tracks.
The most serious impediment to a lasting archive is the evolution of media, platforms, formats, and the applications that create them http://www.acm.org/ubiquity/views/g_bell_1.html (Bell, 2000). Unique, proprietary, and constantly evolving data-formats, e.g. Acrobat-4, MPEG-4, Oracle 8, Quicken 2001, Real G2, Word 2000, etc. suggest or even guarantee obsolescence. The new version may not read legacy data on legacy platforms forever. The basic question is: “How will the data be readable in 10 … 50 years – what are the few, “golden” data formats that we can depend on forever?”
Since CyberAll will store all personal information, e.g. documents, photos, and videos, this data needs to be valid and hence understood in an indeterminate future! High quality paper will hold information for a millennium (or at least several centuries), and film is sometimes rated at several hundred years (if you keep it very cold). A CD is likely to be readable in 50 years, but finding the CD reader/computer and file system/app to read it will clearly be impossible if history is a guide[5]. Is paper the only true long-term store?
Digital documents are committed to a conversion treadmill. With each generation of media (e.g. c1978 8” floppies), the computer system (e.g. CPM), and the application (e.g. Wordstar), a conversion is required. This happens about once a decade, if we pick formats carefully. For plain documents, the alternative is paper stacks of personal information in file cabinets as compulsive info pack rats do today, versus a single DVD that a computer can search!
JPEG is constrained by camera equipment and the need to interoperate. TIF is constrained as a FAX standard. Starkweather’s Pedistil system (Starkweather, 2000) scans documents (e.g. books, journals, papers) into TIF at 400 dpi followed by OCR for retrieval .
Data has learned that to be understood in the future, it cannot be subject to apps that change every year such that a particular version has to be maintained e.g. Quicken 95…2000.[6] As apps evolve, this means data maintains the creating version of the app or all past data associated with a named app has to be converted forward. This is also an issue and perhaps failure of object technology that runs on a single, universal machine.
Alternatively, the one way to ensure interpretability is to transform data emanating from a program, into a generic format. The current solution for longevity is to use a few widely accepted data-types that data is transformed into. For example, yearly Quicken reports end up as .TXT files.
Given the vast amount of data in Adobe’s PDF, or Microsoft’s Office[7] what commitment will the apps make to their data? Will the JPEG and TIF working group ensure that my old files can be read?
Tables 2 gives the storage requirements and costs for various data. The cost of documents and photographs in a CyberAll is nearly zero, hence purging anything is generally a bad strategy!
The cost for storing encoded CDs
is about 1/20th the cost of a CD, not counting the encoding time.
The encoding cost is comparable to a CD cost.
Playback appliances and personal computers are likely to change the
music distribution industry, making CDs obsolete within a decade.
|
Table 2. Storage requirements and cost
for common data items |
||||
|
Item |
Size (Bytes) |
Encoded size |
Items/GByte |
Cost ($)/item* |
|
page (b/w) fax |
100 K |
4K |
10 - 250 K |
0.00004 - .001 |
|
page (color) |
6 M |
0.3(jpeg) |
160 –3 ,500 |
0.003 - 0.06 |
|
business card |
5 K |
500 |
200 K |
0.00005 |
|
Photograph |
3 M |
25-400 K |
10,000 |
0.001 |
|
book
350 pp |
25 M |
1-2 M |
40-750 |
0.01 - 0.25 |
|
|
|
|
|
|
|
CD
(1 hr) |
640 M |
60 M |
1.5 -16 |
$0.60 |
|
|
|
|
|
|
|
LowQ
video/hr |
50-300 K/bs |
20-300 M |
3.3 - 50 |
0.002 – 3.30 |
|
Mpeg
video/hr |
1.5 Mb/s |
670 M |
1.5 |
6.70 |
|
HiQ
video/hr |
DVD 4 Mb/s |
1.8 G |
0.6 |
18 |
*2000 system prices of $10,000 per terabyte or $10 per Gigabyte
Table 3 estimates the requirements for storing an individual’s content for life. One will be able to record all of the information accumulated in their entire personal and professional life in a few terabytes, including everything spoken, but not everything captured via video. This archive would include all home videos for most families. The table shows the various jumps in storage required going from recording lifetime text, transcribed or encoded speech, and video. The need to recognize and only handle transcribed speech is clear based on storage and searching needs.
|
Table 3. Size for
storing everything read/written, heard/spoken, photographed and seen (via
video) |
|||
|
Data-types |
Rate |
Per day / |
Lifetime amount |
|
read text, few pictures |
200 K |
2 –10 M/G |
60-300 G |
|
Email, papers, written text |
|
0.5 M/G |
15 G |
|
photos w/voice @100KB |
200 K |
2 M/G |
60 G |
|
photos @200 KB |
Ten photos/day |
2M/2G |
150 G |
|
|
|
|
|
|
spoken text @120wpm |
43 K |
0.5 M/G |
15 G |
|
Spoken text @8Kbps |
3.6M |
40M/40G |
1.2T |
|
|
|
|
|
|
video-lite 50Kb/s POTS |
22 M |
0.25 G/T |
25 T |
|
video 200Kb/s VHS-lite |
90 M |
1 G/T |
100 T |
|
DVD video 4.3Mb/s |
1.8 G |
20 G/T |
1 P |
The actual storage (Table 4) of my CyberAll is quite small. Until recently I purged paper files to stay within cabinet and searching constraints, hence my CyberAll is incomplete. These are 2-15 Mbyte PowerPoint albums of JPEG photos.
|
Table 4. Author’s document, photograph, videotape; and 150 CD archive |
||||
|
What |
Files |
Size(MB) |
MB/file |
MB/Yr |
|
Digital
document archive |
8234 |
1629 |
0.20 |
50 |
|
TIF &
PDF paper scan archive |
3402 |
2788 |
0.82 |
1000 |
|
GB books
(4 encoded) |
2027 |
494 |
0.24 |
|
|
Photos
(all types) |
3920 |
492 |
0.13 |
50 |
|
Photo
Albums (PowerPoint) |
55 |
151 |
2.75 |
|
|
Mail (2.5
years) |
2 |
330 |
165 |
200 |
|
GB Videos
(lectures, 8mm) |
20 |
4000 |
200 |
|
|
150 CDs
MS multimedia @16 KBps |
1497 |
5820 |
3.89 |
600 |
|
|
19157 |
15704 |
|
1900 |
Deciding among the array of programs using mostly proprietary data formats and developing a process to deal with the encoding of documents for personal/professional and archive/working use is probably the most difficult decision in building one’s CyberAll. These formats have to be maintained or converted in the future. One strategy is to wait for an ideal solution and stability that will hopefully come within five years. Alternatively, keeping data in the most primitive, scanned or encoded form, i.e. TIF and JPEG, allows for future flexibility, including being able to utilize better tools e.g. OCR and encoding.
Photo storage and retrieval are certain to improve. Cameras and acquisition software must maintain dates. Cameras should include audio recording for voice annotation, meta-data for retrieval, and improving the value of the image with a bit of sound or voice. Advances in query software to find like images, e.g. buildings, people, or sunsets using color spectra, shapes, and other attributes is improving to the point of usefulness.
It is necessary to widen CyberAll’s scope to include all family members e.g. children in order to get a better handle on everyday archiving and use for non-computer users. My CyberAll operates with and is aided by my file organization. A general system to operate over decades would have to posit a structure and tools to aid users and cope with the inter-generation problem.
CyberAll requires tools that relate to privacy, especially the ability to lock files until time has elapsed or events occur. And the ability to limit visibility of documents to specific people or people in specific roles.
In the future, retaining conversations and video are possible. Device to record interviews and meetings would be welcome and necessary. When conversations are added, ownership and privacy become even more complex.
Video will be the focus for many users’ CyberAll. Waiting for better editing, searching (especially using recognized text),and retrieval tools and even larger disks is my strategy.
Already, the project has convinced me that a goal of paperless storage and transmission is attainable now for everything except books and items that represent money.
In 2000 the cost to store all personal and professional related, computer generated and paper forms of information, including CDs and photographs is nil, especially compared with physical objects. It is costly to load a personal store and to maintain it for the indefinite future. Information is held in multiple formats to increase the likelihood for long-term retrievability.
Scanners to TIF with and text OCR will make paper input as easy as discarding it. Thus, a state of paperless storage and transmission is near. Standards and ease of use are now the key enabling technologies.
In the next five years, anyone will be able to have a personal computer that retains everything they’ve read, written, and presented via video that came from a computer or legacy source such as paper or videotape. This would include all of the transactions for a family (e.g. correspondence, and every conceivable medical and financial record).
In ten years, systems should be able to recall every personal lifetime conversation. Currently, significant effort is required to build and utilize such systems if they involve the entry of legacy documents, e.g. books, papers, photos, and videotape. A system like CyberAll should be a “killer app” for personal computers.
The author is indebted to colleagues at Microsoft Research for assistance and ideas. Robert Eberl helped navigate the scanner morass. Jim Gray deserves special thanks for reading and revision – and special blame for failing to convince me that a database system will solve all my problems. Vicki Rozycki is creating CyberAll. Diane Crawford reset provided ideas and the critical revision constraints.
Bell, C.G. http://research.microsoft.com/~gbell
co-authored books, papers, talks, videotaped lectures, trip albums, CyberMuseum
items, etc.
Bell, G. “Dear Appy, How Committed are you? Signed Lost and Forgotten Data”, ACM Ubiquity, 21 February 2000, Issue 1. http://www.acm.org/ubiquity/views/g_bell_1.html
Bush, V. As We May Think, Atlantic Monthly, July 1945.
DjVu www.DjVu.com
ECabinet – Ricoh Corporation, Product Brochure; www.ricoh-usa.com Press Release (November 1999)
Gates, B. The Road Ahead.
Penguin Books, 1996.
M. N. Huhns, and L. N.
Stephens, Personal Ontologies, IEEE Internet Computing, Vol 3. No. 5,
September/October 1999, pp p85-89.
Kahle, Brewster (2000). http://www.archive.org
Lesk, M. Practical Digital Libraries. San Francisco: Morgan Kaufmann Publishers, 1997.
PhotoBase - www.arcsoft.com/
Starkweather, G. K. “Pedistal: A Personal Document Imaging System”, June 2000.
[1] : This work has been submitted for publication. Copyright may be transferred without further notice and the accepted version may then be posted by the publisher.
[2] http://www.research.microsoft.com/~gbell/CyberMuseumPubs.htm holds items of historical interest e.g. Hollerith’s Patent, various Digital Equipment Corp. manuals, posters, photos, a talk about Seymour Cray.
[3] The financial community – hiding behind “user resistance” is decades behind in dealing with items as pure bits! The June 2000 law approving electronic signatures eliminates one more barrier.
[4] PDF is a current, significant de facto standard claiming roughly a billion documents implying a total capacity of at least 100 Terabytes.
[5] 3000 of the author’s c1975 documents stored on 8” floppy disks created on a Digital PDP-8 Word Processing System were converted into Word format using a PDP-8 emulator running WPS 8 software.
[6] Data written in 1990 on a MAC and converted forward to a more recent version is unable to generate a report. Data written on a MAC cannot be converted across i.e. read on a PC without manual effort. MacDraw … Draw for the PC have similar problems.
[7] Data written in the early 1980s can been converted forward from MAC versions 4.0 and converted across to the PC Office 2000 standards for Excel, PowerPoint, and Word. PowerPoint can be converted to JPEG or HTML.