Archive Team
{{Short description|Group dedicated to web archiving and digital preservation}}
Archive Team is a group dedicated to digital preservation and web archiving that was co-founded by Jason Scott in 2009.
Its primary focus is the copying and preservation of content housed by at-risk online services. Some of its projects include the partial and complete preservation of services such as GeoCities, Yahoo! Video, Google Video, Friendster, FortuneCity,{{efn|name=ArchiveTeamSuccessStories}} TwitPic, SoundCloud, and the "Aaron Swartz Memorial JSTOR Liberator". Archive Team also archives URL shortener services and wikis on a regular basis. The content archived by Archive Team is usually made available in the Wayback Machine, that is the recommended way of accessing it.{{Cite web |title=Frequently Asked Questions - Archiveteam |url=https://wiki.archiveteam.org/index.php/Frequently_Asked_Questions |access-date=2025-05-26 |website=wiki.archiveteam.org}}
According to Jason Scott, "Archive Team was started out of anger and a feeling of powerlessness, this feeling that we were letting companies decide for us what was going to survive and what was going to die." Scott continues, "it's not our job to figure out what's valuable, to figure out what's meaningful. We work by three virtues: rage, paranoia, and kleptomania."
Warrior/Tracker system
File:Archive-team-warrior-virtual-machine-ui.webp
Archive Team is composed of a loose community of independent contributors/users.{{cite web |last1=Wodinsky |first1=Shoshana |last2=Mehrotra |first2=Dhruv |date=9 April 2021 |title=We’re Archiving Yahoo Answers So You’ll Always Know How Babby Is Formed |url=https://gizmodo.com/were-archiving-yahoo-answers-so-youll-always-know-how-b-1846643969 |website=Gizmodo |url-status=live |archive-url=https://web.archive.org/web/20250124105201/https://gizmodo.com/were-archiving-yahoo-answers-so-youll-always-know-how-b-1846643969 |archive-date=24 January 2025 |access-date=13 April 2025}}{{cite web |last=Hill |first=Mark |date=12 May 2021 |title=Meet the Activist Archivists Saving the Internet From the Digital Dustbin |url=https://www.discovermagazine.com/technology/meet-the-activist-archivists-saving-the-internet-from-the-digital-dustbin |website=Discover Magazine |url-status=live |archive-url=https://web.archive.org/web/20241213061108/https://www.discovermagazine.com/technology/meet-the-activist-archivists-saving-the-internet-from-the-digital-dustbin |archive-date=13 December 2024 |access-date=13 April 2025}}{{cite web |last=Mühlenmeier |first=Lennart |date=26 July 2023 |title=Shutdowns don’t stop during the weekends |url=https://netzpolitik.org/2023/archive-team-shutdowns-dont-stop-during-the-weekends/ |website=netzpolitik.org |url-status=live |archive-url=https://web.archive.org/web/20250329180122/https://netzpolitik.org/2023/archive-team-shutdowns-dont-stop-during-the-weekends/ |archive-date=29 March 2025 |access-date=13 April 2025}} Their archival process makes use of a "Warrior", a virtual machine environment. Individuals use the Warrior in their desktop environments to download content without requiring technical expertise. Tasks are allocated by a centrally-managed Tracker that networks with and allocates items to Warriors. The tracker also monitors user upload activity and displays a leader board.
= Warrior Projects =
There are several long-running Warrior projects:
- Imgur: The image host Imgur updated their terms of service on April 19, 2023. This update focused on removing old, unused, and inactive content that is not tied to a user account, along with NSFW content.
- Blogger: In May 2023, Google announced that inactive accounts would be deleted starting on 2023-12-01 across their platform, including Blogger blogs.
- Reddit: Banning communities that generate bad PR for Reddit Inc. Restricting access to APIs and data on June 19, 2023.
- Russian invasion of Ukraine: Archiving various .ua sites in the wake of the Russian government's invasion.
- Telegram: Archiving public messages in various newsworthy and/or otherwise notable Telegram channels.
- GitHub: When it was bought by Microsoft in 2018, many archivists and users were worried the site would become more restrictive. This project archives the UI parts of GitHub and the code of each repository.
- Mediafire: On 2020-12-18, users reported that they began receiving emails from MediaFire how they plan to classify accounts as abandoned if they fail to meet certain criteria, starting in January.
- Coronavirus Outbreak: Documenting and preserving data, events, and impacts of COVID-19 on society.
- YouTube: Saving metadata, thumbnails, comments and selected videos. Videos and channels are to be limited to: Channels that may be deleted because company went bankrupt, channel owner died, YouTube banning certain content, and channels related to world events and politics.
- Wikiteam: Saving wiki xml dumps.
- Urlteam: Saving URL shorteners.
- URLs: Archiving URLs from various sources.
{{As of|2024|December|12}}, the largest project on ArchiveTeam is URLs, with over 10 petabytes archived.{{efn|The tracker uses units with binary prefixes e.g. pebibytes (1024 TiB, ~1126TB) instead of petabytes (1000 TB)}}
ArchiveBot
ArchiveBot is a web archiving system operated by the Archive Team for conducting curated crawls of websites. Controlled through an IRC channel, ArchiveBot allows volunteers to submit URLs for archiving, typically in response to site shutdowns, policy changes, or other events threatening online data.
Jobs are processed by a network of worker systems known as pipelines, which crawl and save content in the WARC (Web ARChive) format. Volunteers monitor active crawls (jobs) via a public dashboard and may apply ignore rules to handle problematic areas of websites—such as calendars, infinite scroll, or session-based content that can disrupt recursive crawling.{{Cite web |title=ArchiveBot - Archiveteam |url=https://wiki.archiveteam.org/index.php/ArchiveBot |access-date=2025-05-27 |website=wiki.archiveteam.org}}
The results of ArchiveBot crawls are uploaded to the Internet Archive and are typically accessible through the Wayback Machine, where they can be viewed by the public.{{Cite web |title=ArchiveBot: The Archive Team Crowdsourced Crawler |url=https://archive.org/details/archivebot |access-date=2025-05-27 |website=archive.org}} ArchiveBot has been used to preserve a wide range of content, including user-generated platforms, news outlets, and government websites.{{Cite web |title=Domains - ArchiveBot Viewer |url=https://archive.fart.website/archivebot/viewer/domains |access-date=2025-05-27 |website=archive.fart.website}}
See also
{{div col|colwidth=30em}}
- Anna's Archive
- Digital Dark Age
- Digital hoarding
- Flashpoint Archive
- Internet Archive
- List of digital preservation initiatives
- Wayback Machine
{{div col end}}
Notes
References
{{Reflist|30em|refs=
{{Cite web|title=Open Source Bridge 2012 Keynote - Jason Scott| website=YouTube | date=28 June 2012 |url=https://www.youtube.com/watch?v=tJqZGRIwtxk#t=1242s|access-date=2018-11-28|archive-url=https://web.archive.org/web/20170914044651/https://www.youtube.com/watch?v=tJqZGRIwtxk#t=1242s|archive-date=2017-09-14|url-status=live}}
{{Cite web|title=Open Source Bridge 2012 Keynote - Jason Scott| website=YouTube | date=28 June 2012 |url=https://www.youtube.com/watch?v=tJqZGRIwtxk#t=703s |access-date=2018-11-28|archive-url=https://web.archive.org/web/20170914044651/https://www.youtube.com/watch?v=tJqZGRIwtxk#t=703s |archive-date=2017-09-14|url-status=live}}
}}
External links
- {{official website|http://www.archiveteam.org}}
- [https://archive.org/details/archiveteam Archive Team collection] at Internet Archive
- {{Twitter}}
- {{YouTube|id=-2ZTmuX3cog|title=ARCHIVE TEAM: A Distributed Preservation of Service Attack}} by Jason Scott
- [https://www.reddit.com/r/ArchiveTeam ArchiveTeam subreddit] at reddit.com
{{Jason Scott}}
Category:Organizations established in 2009