FlexReadDir: up to 3.8× faster directory listing in Go

说明与用途

FlexReadDir 通过消除每个文件的系统调用并减少分配，显著加速 Go 语言中的目录列表，最高可提升 3.8 倍。

问题

Directory listing in Go is slow because it performs an extra system call per file on macOS and Linux, leading to many allocations and high latency for tools that scan directories frequently.

结果

On macOS, FlexReadDir is 3.8× faster than os.ReadDir+Info for 1,000 files; on Linux 1.6× faster; on Windows 1.3× faster, as shown by median benchmark results.

工作流

• Install the library with go get github.com/pbednarcik/flexreaddir/frd
• Call frd.ReadDir(".") to obtain a sorted slice of Entry structs
• Iterate over entries, using e.Name(), e.Size(), e.ModTime() for processing
• Compare current entries with previous scan using == on Entry structs to detect changes

证据

On macOS and Linux, that's one call to list the directory, then one lstat for every file.

来源

FlexReadDir here is AppendDir, unsorted and reusing its result slice, as a walker would use it.

来源

FlexReadDir is 3.8× faster on macOS, 1.6× on Linux, 1.3× on Windows for 1,000 files.

来源