SSH for loop: parameter passed to function captured in variable is not expanded

Question

I have several servers which have several files with deploy date info I need to parse, and get a local copy of the files which are 2 months old or older.

#!/bin/bash

# split on new line not space
# don't want to quote everything
# globbing desired
IFS=$'\n'

servers=(
  blue
  red
)

parseDate() {
    grep 'deploy_date:' $file | \
    sed  ...                  | \
    # much more in here...
}

getOldFiles() {
    local #all vars
    currentDateInSecs=$(date +%s)
    twoMonthsInSecs=526000
    for server in ${servers[@]}; do
        oldfile=$(
          ssh $server "
            $(declare -f parseDate)
            for file in /dir/$server/file*; do
                dateInSecs=$(date -jf '%b/%e/%Y' $(parseDate) +%s)
                timeDiff=$((\$currentDateInSecs - \$dateInSecs))
                ((\$timeDiff >= \$twoMonthsInSecs)) &&
                    cat \$file
            done
          "
        )
        [ -n $oldFile ] &&
            cat $oldFile ~/oldFiles/${server}-${file}.txt
    done

Problem:

Current issue is with dateInSecs=$(date -jf '%b/%e/%Y' $(parseDate \$file) +%s).

When parseDate \$file is in $() the $file variable is not expanded, it works fine without command substitution, but I need it.

How do I fix this?

Info:

This isn't a script, they're in my ~/.bash_profile

This is a script (among other scripts) in a git repo which is sourced from ~/bash_profile (I have an install script which sets sources using $PWD) so people can use these commands directly instead of cd'ing to the git repo (which has many other things not applicable to the them).

Runs from Macos to CentOS servers.

Answer 1

Everything that can be expanded inside your oldfile=$(...) command substitution is being expanded before you ssh. You need to escape the various internal $() if you want them to run on the remote instead of the local. Anything you want to be evaluated on the remote, needs escaping. Try:

oldfile=$(
          ssh $server "
            $(declare -f parseDate)
            for file in /dir/$server/file*; do
                dateInSecs=\$(date -jf '%b/%e/%Y' \$(parseDate \$file) +%s)
                timeDiff=\$(($currentDateInSecs - \$dateInSecs))
                ((\$timeDiff >= \$twoMonthsInSecs)) &&
                    cat \$file
            done
          "
        )

Also, while BSD date has the -j option, GNU date does not. Since your are connecting to a CentOS machine, you probably want to change

dateInSecs=\$(date -jf '%b/%e/%Y' \$(parseDate \$file) +%s)

to

dateInSecs=\$(date \$(parseDate \$file) +%s)

But we would need to see the exact format you get from parseDate to be sure.

Answer 2

Be careful with your double and single quotes. Where you use double quotes, all $variables and $( … ) expressions will be evaluated before the command is executed. For example:

ssh $server "
    $(declare -f parseDate)
    for file in /dir/$server/file*; do
        dateInSecs=$(date -jf '%b/%e/%Y' $(parseDate \$file) +%s)
        timeDiff=$((\$currentDateInSecs - \$dateInSecs))
        ((\$timeDiff >= \$twoMonthsInSecs)) && cat \$file
    done
"

Here, if $server is remoteHost, this will be prepared as something like this before being passed through ssh to the remote $server:

ssh remoteHost "
    parseDate() { … }
    for file in /dir/remoteHost/file*; do
        dateInSecs=
        timeDiff=
        ((\$timeDiff >= \$twoMonthsInSecs)) && cat \$file
    done
"

And on the remoteHost itself, you'll try to execute something like this:

    parseDate() { … }
    for file in /dir/remoteHost/file*; do
        dateInSecs=
        timeDiff=
        (( >= )) && cat $file
    done

If you can amend your question to include an explanation of what you're trying to achieve, along with example input and output, someone here may be able to suggest how to rewrite your code to fit the requirements.