Cointime

Download App
iOS & Android

DeepSeek open-sources and releases 3FS, a high-speed parallel file system that optimizes AI data access

On the fifth day of Open Source Week, its Fire-Flyer file system (3FS) was officially open sourced. As a high-performance parallel file system, 3FS can fully utilize modern SSDs and RDMA networks to achieve high-speed data access, improving the efficiency of AI model training and inference.

Key performance indicators of 3FS:

Achieving a total read throughput of 6.6 TiB/s in a 180-node cluster;

Achieving a throughput of 3.66 TiB/minute in a 25-node GraySort benchmark test;

Peak throughput of over 40+ GiB/s for single-node KVCache queries.

3FS adopts a modular architecture, supporting data preprocessing, dataset loading, checkpoint storage/recovery, embedded vector search, and inference KVCache queries, with strong consistency semantics. DeepSeek also synchronously launched the Smallpond data processing framework to further optimize 3FS data management capabilities.

Comments

All Comments