Closed

Research distributed file storage systems on linux on Azure

I want to research a good option for (almost) infinitely scalable file storage system that is fast to use by many concurrent nodes. Nodes are running Ubuntu 20.04. File system needs to be highly fault tolerant and it should handle automatic backups by configuration.

System needs to support basic POSIX file permissions.

There are two options:

1. Use managed (remote) service such as Azure Files (SMB). Even within Azure network the latencies are too high for high performance applications. This option would need a local RAM cache, which I believe is supported already by cachefilesd.

2. Manage the cluster by yourself using open-souce software such as:

- MooseFS (free version has a single point of failure)

- BeeGFS

- LizardFS

- Ceph

- Quobyte

Requirements are:

- POSIX file permissions

- High availability

- Fault tolerant

- Built-in backups via fault tolerance or a tool to use

- High amount of concurrent reads, almost none concurrent writes

What I need to research the best option based on which is easier to setup/manager, performance and especially high availability and fault tolerance. Cluster should be able to handle random node crashes.

I want benchmarks for each good option from you. I also want benchmarks against standard HDD and SDD to get a baseline.

Skills: Linux, Ubuntu, Network Administration, Cloud Computing, System Admin

See more: file storage script download, custom file storage, php file upload bar linux, custom user file storage, website multi user file management systems, online file storage project requirements management manager, secure file storage linux, java distributed file storage, distributed file storage system, azure file storage, azure file storage api, azure file storage api c#, create directory in azure file storage c#, upload file to azure file storage c#, azure file storage class kubernetes, create file in azure file storage c#, azure file storage rest api example, download file from azure file storage, azure file storage pricing

About the Employer:
( 0 reviews ) Kajaani, Finland

Project ID: #28310720

3 freelancers are bidding on average €200 for this job

android2014213

HI There, I am experienced in CEPH with Gcloud. I have deployed erpnext in gcloud with ceph. I can help you in your project. Thanks Ashish A.

€250 EUR in 7 days
(32 Reviews)
5.2
zahranahamed

Hi, I can help you with the required research and also help you install a NDFS. Able to start immediately contact through chat.

€150 EUR in 1 day
(19 Reviews)
4.2
iswaradrmwn

Hi, I've read your requirements and I have proficiency in building Ceph cluster on RBD, CephFS and Ceph RGW use case. From your case, I think you need CephFS and I can help you set it up. Hit me up on PM and let's dis More

€200 EUR in 7 days
(8 Reviews)
2.9