-1

This is a continuation of my previous question Bash: Choose row with the highest value in specified column in multiple files and create one output file with rows containing the max

The solution I used is:

for file in mi*
do
    sort -k14,14 -n $file | tail -1
done > result

I am wondering if there is an easy way to edit this code so that when I create the output file I can add a column that contains the file names, or if I need to use a different approach.

I found this question Bash Command Get Data from multiple files and append the file name but I am not sure how to adapt it for my needs

Update with more info: files are tsv and have column names

2 Answers 2

0
for file in mi*
do
    printf "%s " $file
    sort -k14,14 -n $file | tail -1
done > result

Or

for file in mi*
do
    echo "$file $(sort -k14,14 -n $file | tail -1)"
done > result

Depending on your delimiter you can replace the space after the filename with a different character.

2
  • The first solution appears to work well except for the first line. The first line prints file_name1, file_name2, all columns as expected. I do have column names if that is part of the problem. The second solution prints the first file name by itself on one line but all following rows are as expected.
    – keenan
    Commented Dec 29, 2021 at 21:43
  • @keenan you should look at the content of that file and run the command from the loop with this file name and try to understand what's going on. Don't look at this site as something that will do all the work for you, but as something you can learn from. Try to make some effort. We don't see all of your files from here. We gave you 95% of the solution, and 5% are corner cases that are for you to check.
    – aviro
    Commented Dec 29, 2021 at 22:00
0

The trick is to put what would have been sent to stdout into a variable and then output that variable and the filename.

You can change the order of the variables in the echo and also add a delimiter if that is significant.

eg.

for file in mi*
do
    myline=$(sort -k14,14 -n $file | tail -1)
    echo "$file $myline"
done > result

The answer you found uses a command named grep for searching; if searching multiple files, grep can optionally prepend the filename where the searched string was found.

2
  • The first line prints out just file name1 with no other columns but all the other rows are as expected. I do have column names which may be part of the problem?
    – keenan
    Commented Dec 29, 2021 at 21:43
  • Can you paste the output ot the following command wc mi* into your question.
    – X Tian
    Commented Dec 30, 2021 at 2:31

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.