ZeroFS vs Amazon S3 Files: Architecture, Cost, and Ops Impact

Architectural contrast

Both services expose a POSIX‑compatible filesystem backed by Amazon S3, yet they take opposite approaches to bucket layout. Amazon S3 Files builds on Amazon EFS; every pathname maps one‑to‑one with an S3 key, and changes are mirrored to a hidden high‑performance tier before being exported asynchronously back to the bucket [1]. ZeroFS stores metadata in an LSM tree and packs compressed, client‑side‑encrypted extents into immutable segment objects. The bucket therefore contains an opaque layout that only ZeroFS can interpret [2].

Write path and durability

When a client writes through the NFS mount, S3 Files writes immediately to the high‑performance tier, guaranteeing durability on the first write. Export to the S3 key starts only after ~60 seconds of inactivity – fsync does not force immediate visibility [1][3]. ZeroFS, accessed over 9P or NFS, uploads sealed data segments and flushes LSM metadata during fsync; once the object store acknowledges the segment, the file is recoverable without a second export step [2].

Read semantics and cold access

A first read on S3 Files may trigger a directory import: metadata for every object is listed, and files smaller than the import threshold (128 KiB by default) are copied into the high‑performance tier. Large objects ≥1 MiB are read directly from S3, but the initial directory scan can take several seconds [1]. ZeroFS relies on a local RAM‑disk and SSD cache. On a cold miss it looks up extents in the LSM tree, issues ranged GETs for the required segment frames, and prefetches adjacent frames. No bulk import occurs, so a working set that fits in cache generates far fewer S3 requests.

S3 API interoperability

Because S3 Files maintains a one‑to‑one mapping, any object written via the mount becomes a normal S3 object after export. Tools such as aws s3api get-object can retrieve the file, and external changes flow back into the filesystem. ZeroFS never exposes the mounted files through the S3 API; the bucket only holds encrypted segment objects and metadata, making the data invisible without the ZeroFS client and its encryption password [2].

Cost considerations

Both models incur standard S3 storage and request fees, but S3 Files adds a high‑performance storage tier billed at $0.30 / GB‑month for resident data, plus $0.03 / GB for reads and $0.06 / GB for writes to that tier [1]. ZeroFS pays only for object storage, GET/PUT requests, and the compute/cache node(s). Compression can halve payload size, reducing S3 GET costs, while the node itself requires ~2 GB RAM plus SSD capacity for the cache. An illustrative 10 TB scenario shows ZeroFS at roughly $115 – $230 per month (depending on compression) versus $3 530 for S3 Files when all data is imported into the high‑performance tier [1].

Operational impact

• Visibility latency – S3 Files cannot guarantee immediate S3‑API availability; ZeroFS never provides it. Choose S3 Files only if downstream services need the objects in their original form.
• Rename and metadata churn – S3 lacks atomic rename, so S3 Files copies each affected object and deletes the source, taking minutes for large trees. ZeroFS updates only its LSM directory entries, avoiding bulk copies.