DEV Community

loading...

Duplicate file finder with computerized plus human judgement

patarapolw profile image Pacharapol Withayasakpunt ・1 min read

In case someone want to reuse my code.

It involves

  • Generating a SQLite file, and using it
  • File extension comparison
  • Filesize comparison up to 3 significant digits (Number.prototype.toExponential(2))
  • Generating SHA256 hash for more similarity sensitive files

I haven't yet put in computer vision for image similarities.

GitHub logo patarapolw / dedupe

Dedupe files by human judgement

dedupe

Duplicate file finder.

Please use human judgement to reduce false positives.




Discussion

pic
Editor guide