Write the output of a find command within a loop to a file

Question

I have a find command nested within a loop and I want to write the output to a file and be able to see it live as well. But every approach I have tried so far has failed to fulfill this.

My while command is this:

while read -r LINE; do find "$LINE" -name "*Bop*" ; done < /drives/d/dirs_to_search.txt

When I run the command above, I can see the list of matching sub-directories printed into my terminal window. I want to keep seeing this list of matches, but at the same time write them to a file.

And the approaches I have tried so far to write the output of find to a file named matched_subdirs:

while read -r LINE; do find "$LINE" -name "*Bop*" | tee /drives/d/matched_subdirs.txt ; done < /drives/d/dirs_to_search.txt

while read -r LINE; do find "$LINE" -name "*Bop*" ; done < /drives/d/dirs_to_search.txt | tee /drives/d/matched_subdirs.txt

while read -r LINE; do find "$LINE" -name "*Bop*" -print; done < /drives/d/dirs_to_search.txt | tee /drives/d/matched_subdirs.txt

while read -r LINE; do find "$LINE" -name "*Bop*" > /drives/d/matched_subdirs.txt ; done < /drives/d/dirs_to_search.txt

while read -r LINE; do find "$LINE" -name "*Bop*"; done < /drives/d/dirs_to_search.txt > /drives/d/matched_subdirs.txt

Some of those should work. Actually, all of them should work, it's just that some of them would only return the last result because you keep overwriting the file. What actually happens when you try the second approach, for instance? That one should be fine. — terdon
– terdon ♦, Commented Jul 13, 2018 at 12:16
What are you going to do with the output later? It would be safer to use the found pathnames directly from within find to do what you need to de with them. — Kusalananda
– Kusalananda ♦, Commented Jul 13, 2018 at 12:21
@Kusalananda, I intend to use them in an rsync loop--a loop similar to the one in my post above--later on. — vivoru
– vivoru, Commented Jul 13, 2018 at 12:42
@vivoru I see no reason to use find at all then as it's pretty straight forward to get rsync to do the work. I'll update my answer with that as soon as I'm back at a computer. — Kusalananda
– Kusalananda ♦, Commented Jul 13, 2018 at 12:50
What operating system are you using? What you describe is very odd, the tee should work (and does, on my Arch). — terdon
– terdon ♦, Commented Jul 13, 2018 at 13:26

Kusalananda · Accepted Answer · 2018-07-13 15:18:17Z

The commands that you have shown should, as terdon says, work.

An alternative that gets rid of tee:

rm -f /drives/d/matched_subdirs.txt
while IFS= read -r pathname; do
    find "$pathname" -name '*Bop*' -print \
        -exec sh -c 'printf "%s\n" "$@" >>matched' sh {} +
done <dirlist

This would let find print the found pathnames for you to look at, and then use an embedded shell script for writing them to the result file.

I'm a tiny bit worried about what you're intending to do with the pathnames in the output file. If you're intending to use them for looping over later, then it would be better to do so from within find directly. You would have issues with strange filenames containing embedded newlines otherwise.

In comments you say that you intend to use the found pathnames in a loop to run rsync. Calling rsync for each pathnames would be very slow, and you'd be better doing this with rsync directly:

while IFS= read -r pathname; do
    rsync -avR --include='*/' --include='*Bop*' --exclude='*' --prune-empty-dirs "$pathname" target
done <dirlist

Here, dirlist is a file with your directories.

Example:

$ tree
.
|-- dirlist
`-- source
    |-- a
    |   |-- dir-1
    |   |   |-- somefile_Bop_here
    |   |   `-- someotherfile
    |   |-- dir-2
    |   |   |-- somefile_Bop_here
    |   |   `-- someotherfile
    |   `-- dir-3
    |       |-- somefile_Bop_here
    |       `-- someotherfile
    |-- b
    |   |-- dir-1
    |   |   |-- somefile_Bop_here
    |   |   `-- someotherfile
    |   |-- dir-2
    |   |   |-- somefile_Bop_here
    |   |   `-- someotherfile
    |   `-- dir-3
    |       |-- somefile_Bop_here
    |       `-- someotherfile
    `-- c
        |-- dir-1
        |   |-- somefile_Bop_here
        |   `-- someotherfile
        |-- dir-2
        |   |-- somefile_Bop_here
        |   `-- someotherfile
        `-- dir-3
            |-- somefile_Bop_here
            `-- someotherfile

13 directories, 19 files

$ cat dirlist
source/a
source/b/dir-2

(running loop here)

$ tree target
target
`-- source
    |-- a
    |   |-- dir-1
    |   |   `-- somefile_Bop_here
    |   |-- dir-2
    |   |   `-- somefile_Bop_here
    |   `-- dir-3
    |       `-- somefile_Bop_here
    `-- b
        `-- dir-2
            `-- somefile_Bop_here

7 directories, 4 files

I chose to use -R (--relative). Without it, I would have gotten

target
|-- a
|   |-- dir-1
|   |   `-- somefile_Bop_here
|   |-- dir-2
|   |   `-- somefile_Bop_here
|   `-- dir-3
|       `-- somefile_Bop_here
`-- dir-2
    `-- somefile_Bop_here

Stack Exchange Network

Write the output of a find command within a loop to a file

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Write the output of a find command within a loop to a file

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions