![Cover image for Saving Millions of Images Series [Part Five]](https://media.dev.to/cdn-cgi/image/width=1000,height=420,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fnmhc8p2dxqw6wlqfcwo3.gif)
File Systems
Sometimes, while working with A LOT OF IMAGES you will decide to store them on your hardware. It means that you need to find a file system to store your stuff.
You can organize a file storage system for saving images and other media things. There are few options to play with.
Below I'll share links to a few open-source file systems for storing images.
You can use projects that implement file systems for such kinds of needs. Everyone knows storing tons of images is bad inside a single directory or NFS etc.
What is the best solution to store 1 billion images and is open-source?
Possible solutions are:
- GlusterFS
- MongoDB
- SeaweedFS
- MogileFS
SeaweedFS
Testimonials from developers:
“Seaweed-FS is very flexible and pared down to the basics. It was created to store billions of images and serve them fast.”
Link: https://github.com/chrislusf/seaweedfs
MogileFS
Testimonials from developers:
“We use MogileFS. We're small-scale users with less than 8TB and some 50 million files. We switched from storing in Amazon S3 some years ago to better control file names and performance. It's not the prettiest software, but it's very "field-tested", and basically, all users are using it the same way you will be.”
Link: http://danga.com/mogilefs/
Or you can use a (distributed) filesystem like HDFS and prepare dedicated web servers as "filesystem clients" to save uploaded images and service requests. Image metadata is saved in an additional database including the file path information for each image.
If you need more information to read, jump into links.
- Introduction to file systems
- File Systems Overview
- Storing images in Blob vs File System | by Anil Singh | Medium
- Whats the best filesystem for managing millions of images? - Server Fault
- https://docs.microsoft.com/en-us/answers/questions/405528/storing-images-on-file-system-as-compared-to-db.html
- Tracker Ten Storing Images in a Database vs File System
- Content Services Platform
- https://www.reddit.com/r/webdev/comments/e0pgee/is_it_better_to_store_images_in_database_or/
- Store Image to Filesystem
- List of file systems - Wikipedia
- @okhosting/awesome-storage
Saving Millions of Images Series
Top comments (0)