find duplicate files
Find duplicate files first by file size and then MD5 sum, with a progress bar:
find -not -empty -type f -printf "%s\n" \
| sort -rn \
| uniq -d \
| xargs -I{} -n1 find -type f -size {}c -print0 \
| tee \
| pv --line-mode --size $(find . -type f | wc -l) \
| xargs -0 md5sum \
| sort \
| uniq -w32 --all-repeated=separate \
| tee /tmp/duplicates