Amazon S3

Amazon S3 manages data with an object storage architecture{{Cite web |url=https://aws.amazon.com/what-is-cloud-object-storage/ |title=What is Cloud Object Storage? – AWS |date=2019-10-16 |website=Amazon Web Services, Inc. |language=en-US |url-status=live |archive-url=https://web.archive.org/web/20180920181620/https://aws.amazon.com/what-is-cloud-object-storage/ |archive-date=2018-09-20 |access-date=2018-07-09}} which aims to provide scalability, high availability, and low latency with high durability. The basic storage units of Amazon S3 are objects which are organized into buckets. Each object is identified by a unique, user-assigned key.{{cite web|url=http://techblog.aasisvinayak.com/starting-websphere-in-cloud-and-saving-the-data-in-s3/|title=Tech Blog » Starting Websphere in Cloud and saving the data in S3|website=techblog.aasisvinayak.com|archive-url=https://web.archive.org/web/20100312080332/http://techblog.aasisvinayak.com/starting-websphere-in-cloud-and-saving-the-data-in-s3|archive-date=2010-03-12|url-status=dead}} Buckets can be managed using the console provided by Amazon S3, programmatically with the AWS SDK, or the REST application programming interface. Objects can be up to five terabytes in size.{{cite web|url=https://github.com/open-guides/og-aws/blob/master/README.md#s3|title=open-guides/og-aws|website=GitHub|access-date=2018-05-17|archive-url=https://web.archive.org/web/20180103123141/https://github.com/open-guides/og-aws/blob/master/README.md#s3|archive-date=2018-01-03|url-status=live}}{{cite web|url=https://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html|title=Error Responses - Amazon Simple Storage Service|website=docs.aws.amazon.com|access-date=2018-05-21|archive-url=https://web.archive.org/web/20171224010324/http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html|archive-date=2017-12-24|url-status=live}} Requests are authorized using an access control list associated with each object bucket and support versioning{{cite web | url=https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html | title=Using versioning in S3 buckets - Amazon Simple Storage Service }} which is disabled by default.{{cite web|url=https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html|title=Introduction to Amazon S3 - Amazon Simple Storage Service|website=docs.aws.amazon.com|access-date=2018-05-17|archive-url=https://web.archive.org/web/20180512235723/https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html|archive-date=2018-05-12|url-status=live}} Since buckets are typically the size of an entire file system mount in other systems, this access control scheme is very coarse-grained. In other words, unique access controls cannot be associated with individual files.{{cn|date=April 2022}} Amazon S3 can be used to replace static web-hosting infrastructure with HTTP client-accessible objects,{{cite web|url=http://www.bucketexplorer.com/documentation/amazon-s3--how-to-use-Amazon-s3-for-web-hosting.html|title=How to use Amazon S3 for Web Hosting|website=bucketexplorer.com|access-date=2008-05-06|archive-url=https://web.archive.org/web/20080408024708/http://www.bucketexplorer.com/documentation/amazon-s3--how-to-use-Amazon-s3-for-web-hosting.html|archive-date=2008-04-08|url-status=dead}} index document support, and error document support.[http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?WebsiteHosting.html Amazon Simple Storage Service] {{Webarchive|url=https://web.archive.org/web/20110220042555/http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?WebsiteHosting.html |date=2011-02-20 }} Docs.amazonwebservices.com. Retrieved on 2013-08-09.

The Amazon AWS authentication mechanism allows the creation of authenticated URLs, valid for a specified amount of time. Every item in a bucket can also be served as a BitTorrent feed. The Amazon S3 store can act as a seed host for a torrent and any BitTorrent client can retrieve the file. This can drastically reduce the bandwidth cost for the download of popular objects. A bucket can be configured to save HTTP log information to a sibling bucket; this can be used in data mining operations.http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html {{Webarchive|url=https://web.archive.org/web/20141223092055/http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html |date=2014-12-23 }} Server Access Logging There are various User Mode File System (FUSE)–based file systems for Unix-like operating systems (for example, Linux) that can be used to mount an S3 bucket as a file system. The semantics of the Amazon S3 file system are not that of a POSIX file system, so the file system may not behave entirely as expected.{{cite web

|url = https://code.google.com/p/s3ql/wiki/other_s3_filesystems

|title = Comparison of S3QL and other S3 file systems

|access-date = 2012-06-29

|archive-url = https://web.archive.org/web/20120805030057/http://code.google.com/p/s3ql/wiki/other_s3_filesystems

|archive-date = 2012-08-05

|url-status = live

}}

Amazon S3 offers nine different storage classes with different levels of durability, availability, and performance requirements.{{cite web|url=https://aws.amazon.com/s3/storage-classes/|title=Cloud Storage Classes – Amazon Simple Storage Service (S3) – AWS|website=Amazon Web Services, Inc.|access-date=2018-05-17|archive-url=https://web.archive.org/web/20180613200647/https://aws.amazon.com/s3/storage-classes/|archive-date=2018-06-13|url-status=live}}

• Amazon S3 Standard is the default. It is general purpose storage for frequently accessed data.
• Amazon S3 Express One Zone is a single-digit millisecond latency storage for frequently accessed data and latency-sensitive applications. It stores data only in one availability zone.{{Cite web |date=2023-11-28 |title=Announcing the new Amazon S3 Express One Zone high performance storage class {{!}} AWS News Blog |url=https://aws.amazon.com/blogs/aws/new-amazon-s3-express-one-zone-high-performance-storage-class/ |access-date=2023-12-01 |website=aws.amazon.com |language=en-US}}
• Amazon S3 Standard-Infrequent Access (Standard-IA) is designed for less frequently accessed data, such as backups and disaster recovery data.
• Amazon S3 One Zone-Infrequent Access (One Zone-IA) performs like the Standard-IA, but stores data only in one availability zone.
• Amazon S3 Intelligent-Tiering moves objects automatically to a more cost-efficient storage class.
• Amazon S3 on Outposts brings storage to installations not hosted by Amazon.
• Amazon S3 Glacier Instant Retrieval is a low-cost storage for rarely accessed data, but which still requires rapid retrieval.
• Amazon S3 Glacier Flexible Retrieval is also a low-cost option for long-lived data; it offers 3 retrieval speeds, ranging from minutes to hours.
• Amazon S3 Glacier Deep Archive is the lowest cost storage for long-lived archive data that is accessed less than once per year and is retrieved asynchronously.

The Amazon S3 Glacier storage classes above are distinct from Amazon Glacier, which is a separate product with its own APIs.

An object in S3 can be between 0 bytes and 5 TB. If an object is larger than 5 TB, it must be divided into chunks prior to uploading. When uploading, Amazon S3 allows a maximum of 5 GB in a single upload operation; hence, objects larger than 5 GB must be uploaded via the S3 multipart upload API.{{Cite web | url=https://riyanchristy.goseeq.net/how-to-upload-large-files-to-s3-efficiently/ | title=How to Upload Large Files to S3 | date=June 21, 2022 | access-date=June 22, 2022 | archive-date=October 1, 2022 | archive-url=https://web.archive.org/web/20221001213248/https://riyanchristy.goseeq.net/how-to-upload-large-files-to-s3-efficiently/ | url-status=dead }}

• Photo hosting service SmugMug has used Amazon S3 since April 2006. They experienced a number of initial outages and slowdowns, but after one year they described it as being "considerably more reliable than our own internal storage" and claimed to have saved almost $1 million in storage costs.{{cite web|url=https://donmacaskill.wordpress.com/2006/11/10/amazon-s3-show-me-the-money/|title=Amazon S3: Show Me the Money|date=November 10, 2006|work=SmugMug Blog|publisher=SmugMug|access-date=2017-03-03|archive-url=https://web.archive.org/web/20170303130206/https://donmacaskill.wordpress.com/2006/11/10/amazon-s3-show-me-the-money/|archive-date=2017-03-03|url-status=live}}
• Netflix uses Amazon S3 as their system of record. Netflix implemented a tool, S3mper,{{cite web|url=http://techblog.netflix.com/2014/01/s3mper-consistency-in-cloud.html|title=S3mper: Consistency in the Cloud|access-date=2016-05-01|archive-url=https://web.archive.org/web/20160424205437/http://techblog.netflix.com/2014/01/s3mper-consistency-in-cloud.html|archive-date=2016-04-24|url-status=live}} to address the Amazon S3 limitations of eventual consistency.{{cite web|title=Introduction to Amazon S3|url=http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html|website=Amazon|access-date=28 December 2017|archive-url=https://web.archive.org/web/20171225230323/http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html|archive-date=2017-12-25|url-status=live}} S3mper stores the filesystem metadata: filenames, directory structure, and permissions in Amazon DynamoDB.{{Cite web|url=https://www.theguardian.com/technology/2017/feb/02/amazon-web-services-the-secret-to-the-online-retailers-future-success|title=Amazon Web Services: the secret to the online retailer's future success|last=Hern|first=Alex|date=2017-02-02|website=the Guardian|language=en|access-date=2018-04-23|archive-url=https://web.archive.org/web/20180502223750/https://www.theguardian.com/technology/2017/feb/02/amazon-web-services-the-secret-to-the-online-retailers-future-success|archive-date=2018-05-02|url-status=live}}
• Reddit is hosted on Amazon S3.{{cite web |url=http://aws.amazon.com/solutions/case-studies/reddit/ |title=AWS Case Study: reddit |work=aws.amazon.com |year=2015 |access-date=March 18, 2015 |archive-url=https://web.archive.org/web/20150317005440/http://aws.amazon.com/solutions/case-studies/reddit/ |archive-date=2015-03-17 |url-status=live }}
• Bitcasa,{{cite web|url=https://www.bitcasa.com/legal|date=May 16, 2013|title=Bitcasa Legal|access-date=2013-05-16|archive-url=https://archive.today/20130628192701/https://www.bitcasa.com/legal|archive-date=2013-06-28|url-status=dead}} and Tahoe-LAFS-on-S3,{{cite web|url=https://leastauthority.com/products|date=August 21, 2012|title=What is Tahoe-LAFS-on-S3?|access-date=2012-08-21|archive-url=https://web.archive.org/web/20130506021742/https://leastauthority.com/products|archive-date=2013-05-06|url-status=live}} among others, use Amazon S3 for online backup and synchronization services. In 2016, Dropbox stopped using Amazon S3 services and developed its own cloud server.{{Cite magazine|url=https://www.wired.com/2016/03/epic-story-dropboxs-exodus-amazon-cloud-empire/|title=The Epic Story of Dropbox's Exodus From the Amazon Cloud Empire|magazine=WIRED|access-date=2018-04-23|language=en-US|archive-url=https://web.archive.org/web/20180125195150/https://www.wired.com/2016/03/epic-story-dropboxs-exodus-amazon-cloud-empire/|archive-date=2018-01-25|url-status=live}}{{Cite news|url=https://www.geekwire.com/2018/dropbox-saved-almost-75-million-two-years-building-tech-infrastructure/|title=Dropbox saved almost $75 million over two years by building its own tech infrastructure|date=2018-02-23|work=GeekWire|access-date=2018-04-23|language=en-US|archive-url=https://web.archive.org/web/20180423170423/https://www.geekwire.com/2018/dropbox-saved-almost-75-million-two-years-building-tech-infrastructure/|archive-date=2018-04-23|url-status=live}}
• Swiftype's CEO has mentioned that the company uses Amazon S3.{{cite web|url=http://stackshare.io/posts/swiftype-explains-their-cloud-stack|date=July 1, 2013|title=Swiftype Explains Their Cloud Stack|access-date=2014-12-08|archive-url=https://web.archive.org/web/20141208083021/http://stackshare.io/posts/swiftype-explains-their-cloud-stack|archive-date=2014-12-08|url-status=live}}

The broad adoption of Amazon S3 and related tooling has given rise to competing services based on the S3 API. These services use the standard programming interface but are differentiated by their underlying technologies and business models.{{cite web|last=Watters|first=Audrey|title=Cloud Community Debates, Is Amazon S3's API the Standard? (And Should It Be?)|date=12 July 2010|url=http://readwrite.com/2010/07/12/cloud-community-debates-is-ama|publisher=SAY Media, Inc.|access-date=19 December 2012|archive-url=https://web.archive.org/web/20130217035941/http://readwrite.com/2010/07/12/cloud-community-debates-is-ama|archive-date=2013-02-17|url-status=live}} A standard interface enables better competition from rival providers and allows economies of scale in implementation, among other benefits.{{cite book |others=Committee on Standards Workshop Planning, Board on Telecommunications and Computer Applications, Commission on Engineering and Technical Systems, National Research Council|title=Crossroads of Information Technology Standards|year=1990|publisher=The National Academies Press, 1990.|location=Washington, DC|pages=36–37|doi=10.17226/10440|isbn=978-0-309-58171-4|url=http://www.nap.edu/openbook.php?record_id=10440&page=36|access-date=2014-03-25|archive-url=https://web.archive.org/web/20140325160847/http://www.nap.edu/openbook.php?record_id=10440&page=36|archive-date=2014-03-25|url-status=live}}

File:AmazonS3TwoTrillionObjects.JPG announces 2 trillion objects stored in S3.]]

Amazon Web Services introduced Amazon S3 in 2006.Overview of Amazon Web Services, 2018, https://docs.aws.amazon.com/whitepapers/latest/aws-overview/introduction.html {{Webarchive|url=https://web.archive.org/web/20171118025949/https://d1.awsstatic.com/whitepapers/aws-overview.pdf |date=2017-11-18 }}Garfinkel, Simson L. 2007. An Evaluation of Amazon's Grid Computing Services: EC2, S3, and SQS. Harvard Computer Science Group Technical Report TR-08-07. https://dash.harvard.edu/bitstream/handle/1/24829568/tr-08-07.pdf?sequence=1 {{Webarchive|url=https://web.archive.org/web/20180729230848/https://dash.harvard.edu/bitstream/handle/1/24829568/tr-08-07.pdf?sequence=1 |date=2018-07-29 }}

In November 2017 AWS added default encryption capabilities at bucket level.{{Cite web|url=https://www.youtube.com/watch?v=NXehLy7IiPM&list=PL2yQDdvlhXf_ZsP25dGLTNbrVSphM2JDl&index=666|title = AWS re:Invent 2024 - Dive deep on Amazon S3 (STG302)

|date = 9 December 2024}}

Amazon S3 provides a durability guarantee of 99.999999999% (referred to as "11 nines"), primarily addressing data loss from hardware failures. However, this guarantee does not extend to losses resulting from human errors (such as accidental deletion), misconfigurations, third-party failures and subsequent data corruptions, natural disasters, force majeure events, or security breaches. Customers are responsible for monitoring SLA compliance and must submit claims for any unmet SLAs within a designated timeframe. They should understand how deviations from SLAs are calculated, as these parameters may differ from those of other AWS services. These requirements can impose a significant burden on customers. Additionally, SLA percentages and conditions can vary from those of other AWS services. In cases of data loss due to hardware failure attributable to Amazon, the company does not provide monetary compensation; instead, affected users may receive credits if they meet the eligibility criteria.{{Cite book |title=Cloud Security and Privacy An Enterprise Perspective on Risks and Compliance |date=September 4, 2009 |publisher=O'Reilly Media |isbn=9781449379513}}{{Cite book |title=Requirements Engineering for Service and Cloud Computing |date=April 10, 2017 |publisher=Springer International Publishing |isbn=9783319513102}}{{Cite book |last=Srinivasan |title=Cloud Computing Basics |date=May 14, 2014 |publisher=Springer |isbn=9781461476993}}{{Cite book |last=Murugesan |first=San |title=Encyclopedia of Cloud Computing |date=August 2016 |publisher=John Wiley & Sons |isbn=9781118821978}}{{Cite book |title=Genomics in the AWS Cloud Analyzing Genetic Code Using Amazon Web Services |isbn=9781119573401}}

• Amazon Elastic Block Store (EBS)
• Timeline of Amazon Web Services

{{Reflist}}

{{refbegin}}

• {{cite web

| title = Server Access Logging

| url = http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html

}}

• {{cite web

| title = Amazon S3 Developer Guide

| url = http://docs.amazonwebservices.com/AmazonS3/latest/dev/

| date = 2006-03-01

}}

• {{cite web

| title = Amazon S3 Introduces Storage Pricing Tiers

| url = http://developer.amazonwebservices.com/connect/ann.jspa?annID=351

| date = 2008-10-08

}}

• {{cite web

|title = RightScale Ruby library to access Amazon CloudFront, EC2, S3, SQS, and SDB

|url = http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1014

|date = 2007-10-27

|access-date = 2009-01-07

|archive-url = https://web.archive.org/web/20081103070950/http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1014

|archive-date = 2008-11-03

|url-status = dead

}}

{{refend}}

{{-}}

{{Amazon}}

{{Cloud computing}}

Category:Amazon Web Services

Category:Cloud storage

Category:File hosting

Category:Network file systems

de:Amazon Web Services#Amazon Simple Storage Service (S3)