Skip to main content
Unix & Linux

Return to Answer

added 433 characters in body
Source Link
terdon ♦
terdon ♦
  • 252.7k
  • 69
  • 481
  • 719

To include the command in your second example, do:

#!/bin/awk -f
BEGIN{
  FS=","
}
{
  if($1~/^0/){zeros++}
  if($1~/^1/){ones++; sum8+=$8}
}
END{
  printf "No of lines containing 0 on the 1st column: %d\n", zeros;
  printf "No of lines containing 1 on the 1st column: %d\n", ones;
  printf "Sum of all 8th fields where the 1st field starts with 1: %d\n", sum8
}

#!/bin/bash
awk -F"," '($1~/^0/){zeros++}
           ($1~/^1/){ones++}
           END{ 
                printf "No of lines containing 0 on the 1st column: %d\n", zeros;
                printf "No of lines containing 1 on the 1st column: %d\n", ones;
           }' "$1"

Finally, if you really want to keep this as separate commands, you could do something like this but it will be very slow since it needs to read the file multiple times:

#!/bin/bash

echo "No of lines containing 0 on the 1st column: "
awk -F ','  '{print $1}' "$1" | awk '/^0/' | wc -l
echo "No of lines containing 1 on the 1st column:"
awk -F ','  '{print $1}' "$1" | awk '/^1/' | wc -l
echo "Sum of all the 8th columns where the 1st column starts with 1:"
awk -F ','  '/^1/{sum+=$8} END {print sum}' "$1"

You would then make the file executable (chmod a+x /path/to/foo.sh) and run it like this:

/path/to/foo.sh /path/to/test.csv

#!/bin/bash
awk -F"," '($1~/^0/){zeros++}
           ($1~/^1/){ones++}
           END{ 
                printf "No of lines containing 0 on the 1st column: %d\n", zeros;
                printf "No of lines containing 1 on the 1st column: %d\n", ones;
           }' "$1"

To include the command in your second example, do:

#!/bin/awk -f
BEGIN{
  FS=","
}
{
  if($1~/^0/){zeros++}
  if($1~/^1/){ones++; sum8+=$8}
}
END{
  printf "No of lines containing 0 on the 1st column: %d\n", zeros;
  printf "No of lines containing 1 on the 1st column: %d\n", ones;
  printf "Sum of all 8th fields where the 1st field starts with 1: %d\n", sum8
}

#!/bin/bash
awk -F"," '($1~/^0/){zeros++}
           ($1~/^1/){ones++}
           END{ 
                printf "No of lines containing 0 on the 1st column: %d\n", zeros;
                printf "No of lines containing 1 on the 1st column: %d\n", ones;
           }' "$1"

Finally, if you really want to keep this as separate commands, you could do something like this but it will be very slow since it needs to read the file multiple times:

#!/bin/bash

echo "No of lines containing 0 on the 1st column: "
awk -F ','  '{print $1}' "$1" | awk '/^0/' | wc -l
echo "No of lines containing 1 on the 1st column:"
awk -F ','  '{print $1}' "$1" | awk '/^1/' | wc -l
echo "Sum of all the 8th columns where the 1st column starts with 1:"
awk -F ','  '/^1/{sum+=$8} END {print sum}' "$1"

You would then make the file executable (chmod a+x /path/to/foo.sh) and run it like this:

/path/to/foo.sh /path/to/test.csv
Source Link
terdon ♦
terdon ♦
  • 252.7k
  • 69
  • 481
  • 719

OK, first of all, you don't want to do this. Awk is orders of magnitude faster than the shell, so there is no benefit whatsoever in converting an awk script into a shell script! Forget the shell, just do everything in awk. Save this file as foo.awk:

#!/bin/awk -f
BEGIN{
  FS=","
}
{
  if($1~/^0/){zeros++}
  if($1~/^1/){ones++}
}
END{
  printf "No of lines containing 0 on the 1st column: %d\n", zeros;
  printf "No of lines containing 1 on the 1st column: %d\n", ones;
}

Make the file executable with chmod a+x foo.awk and then run it:

/path/to/foo.awk /path/to/test.csv

If I run it on your example data, I get:

$ foo.awk test.csv 
No of lines containing 0 on the 1st column: 4
No of lines containing 1 on the 1st column: 2

If you must use a shell script for some reason, then have the shell script run the awk and nothing else. Do not try to split the input in the shell, that's complicated and very slow. Something like this is much better:

#!/bin/bash
awk -F"," '($1~/^0/){zeros++}
           ($1~/^1/){ones++}
           END{ 
                printf "No of lines containing 0 on the 1st column: %d\n", zeros;
                printf "No of lines containing 1 on the 1st column: %d\n", ones;
           }' "$1"