This post is about cloud-based data storage. Simple Storage Service (S3) is one of the most used cloud services on Amazon AWS. There are many reasons why a company might want to store data in the cloud, such as for building cloud-native applications, for archives, or even for hosting static websites.
S3 provides object storage within containers called buckets. Think of buckets like directories or folders on your hard drive, each having a unique name. There is no limit to the number of objects you can store in a bucket, or on the bucket storage size. However, there is an individual size limit on objects, currently 5TB.
A few advantages to storing files in the cloud include availability, scalability, security, and cost. I find S3 useful to store project files that I need to access from multiple locations (e.g., office, home, field). S3 also makes it easy to maintain archives of old projects. Be aware though that corporate policies or regulations may prohibit certain files from being stored in the cloud. Each company’s situation will be a little different.
As an object storage service, S3 is well suited for static files or files that rarely change like images, videos, backups, basemaps, or large PDFs. S3 is also a useful for storing raw unstructured data at scale. A typical Forestry company might use S3 to store GIS data, inventory files, project files, database backups, contracts, leases, and property deeds. Forestry companies can maintain years of paper-based records in file cabinets. Converting these documents to digital files for upload to S3 might be worth consideration, especially if storage space is limited. Where security is a concern, S3 provides multiple levels of security including access control, bucket policies, and encryption.
As a developer, I like that I have the ability to work with S3 programmatically. Python Boto3 SDK gives programmers full control over creating and maintaining buckets, changing security policies, and uploading or downloading objects. This dramatically increases productivity by automating workflows and running tasks in the background. Of course, AWS provides a web-based interface for easy access to S3 if writing code or working from the command line is not your thing.
Post Photo by Jessica Johnston on Unsplash
Categories: Python Record Storage Technology Adoption
