Conference proceeding
InterPlanetary Wayback: Peer-To-Peer Permanence of Web Archives
RESEARCH AND ADVANCED TECHNOLOGY FOR DIGITAL LIBRARIES, TPDL 2016, v 9819, pp 411-416
01 Jan 2016
Abstract
We have integrated Web ARChive (WARC) files with the peer-to-peer content addressable InterPlanetary File System (IPFS) to allow the payload content of web archives to be easily propagated. We also provide an archival replay system extended from pywb to fetch the WARC content from IPFS and re-assemble the originally archived HTTP responses for replay. From a 1.0GB sample Archive-It collection of WARCs containing 21,994 mementos, we show that extracting and indexing the HTTP response content of WARCs containing IPFS lookup hashes takes 66.6min inclusive of dissemination into IPFS.
Metrics
Details
- Title
- InterPlanetary Wayback: Peer-To-Peer Permanence of Web Archives
- Creators
- Mat Kelly - Old Dominion UniversitySawood Alam - Old Dominion UniversityMichael L. Nelson - Old Dominion UniversityMichele C. Weigle - Old Dominion University
- Contributors
- N Fuhr (Editor)L Kovacs (Editor)T Risse (Editor)W Nejdl (Editor)
- Publication Details
- RESEARCH AND ADVANCED TECHNOLOGY FOR DIGITAL LIBRARIES, TPDL 2016, v 9819, pp 411-416
- Series
- Lecture Notes in Computer Science
- Publisher
- Springer Nature
- Number of pages
- 6
- Grant note
- 1526700 / Div Of Information & Intelligent Systems; National Science Foundation (NSF); NSF - Directorate for Computer & Information Science & Engineering (CISE)
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Information Science
- Web of Science ID
- WOS:000389021000035
- Scopus ID
- 2-s2.0-84984787984
- Other Identifier
- 991021786583904721
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Web of Science research areas
- Computer Science, Information Systems
- Computer Science, Theory & Methods
- Information Science & Library Science