FlexReadDir: up to 3.8× faster directory listing in Go

Описание и применение

FlexReadDir ускоряет чтение каталогов в Go до 3,8× быстрее, устраняя лишние системные вызовы и выделения памяти.

Задача

Directory listing in Go is slow because it performs an extra system call per file on macOS and Linux, leading to many allocations and high latency for tools that scan directories frequently.

Результат

On macOS, FlexReadDir is 3.8× faster than os.ReadDir+Info for 1,000 files; on Linux 1.6× faster; on Windows 1.3× faster, as shown by median benchmark results.

Рабочий процесс

• Install the library with go get github.com/pbednarcik/flexreaddir/frd
• Call frd.ReadDir(".") to obtain a sorted slice of Entry structs
• Iterate over entries, using e.Name(), e.Size(), e.ModTime() for processing
• Compare current entries with previous scan using == on Entry structs to detect changes

Доказательства

On macOS and Linux, that's one call to list the directory, then one lstat for every file.

Источник

FlexReadDir here is AppendDir, unsorted and reusing its result slice, as a walker would use it.

Источник

FlexReadDir is 3.8× faster on macOS, 1.6× on Linux, 1.3× on Windows for 1,000 files.

Источник