Skip to main content
Unix & Linux

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

5
  • Sounds perfect, thanks!! I'll try it and accept if it works as described! Commented Oct 12, 2010 at 20:09
  • 3
    This did exactly what I asked for. However I believe that ZFS with dedup will eventually be the way to do, since I did find that the files had slight differences so only a few could be hardlinked. Commented Dec 8, 2010 at 20:13
  • 15
    Upvoted this, but after researching some more, I kind of which I didn't. rdfind is available via the package managers for ALL major platforms (os x, linux, (cyg)win, solaris), and works at a blazing native speed. So do check out the answer below. Commented Jan 3, 2015 at 13:42
  • @oligofren I was thinking the same, but then I hit [Errno 31] Too many links. This scrips seems to be the only thing that handles that. Commented Jun 26, 2015 at 6:59
  • 8
    Checksumming every single file, rather than only files where there exists at least one other with identical size, is unnecessarily inefficient (and unnecessarily prone to hash collisions). Commented Feb 1, 2016 at 16:56