I would like to run a Python script in my home directory that pulls data from various files. However, to access these files I need to log into an SSH and then navigate to a certain directory. Is there a way to do this while running my script in my home directory?
I would just copy them over using scp, but there are thousands, so I don't think this would be very efficient (but is a last resort option). Thanks!
I would recommend using rsync. This will ssh you in and perform r-sync on whatever directory you want. It's (probably) the most efficient and reliable way to pull a lot of large files.
rsync -a --progress username@remote_host:destination_directory ~/dir1
You can try using sshfs. With it you can mount a file system in your local computer accessing it through your ssh connection.
sudo mkdir /mnt/droplet
sudo sshfs -o allow_other,defer_permissions [email protected]:/ /mnt/droplet
Check this tutorial from digitalocean: https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh
