Built a duplicate file finder in Java, here's how it works

Description and use

DupeLens is a Java CLI that detects duplicate files using size pre-filtering and SHA-256, reports duplicates in the terminal and as an optional HTML report, and does not modify files.

Problem

Need a way to identify duplicate files in a directory without risking accidental deletion of important files.

Outcome

DupeLens finds duplicate files by size pre-filtering and SHA-256 content hashing, reports duplicates in terminal and as an optional HTML report, and does not modify files.

Workflow

• Check file sizes; skip comparison if sizes differ.
• For files with matching sizes, compute SHA-256 checksums to compare contents.
• Display duplicate groups in the terminal.
• Optionally generate an HTML report via --html flag.

Evidence

It's basically a Java CLI that finds duplicate files in your folders and generates an HTML report.

Source

First, it checks file sizes. If two files have different sizes, there's no reason to compare them further.

Source

For files with matching sizes, it uses SHA-256 to check whether their contents are identical.

Source

Once it's done, it shows the duplicates in the terminal. You can also generate an HTML report if you want something easier to look through.

Source

It doesn't delete, move, or modify your files.

Source

You'll need Java 17 or newer.

Source

Download the JAR from GitHub and run: java -jar dupelens.jar ./your-folder --html report.html

Source

GitHub: https://github.com/illusionoftalha/dupelens

Source

Just released v1.0.0, so it's still pretty early.

Source