0

I am writing scripts to poll a large number of remote hosts over ssh using gnu parallel. I copy the script to them, run it and get the results. Right now I'm using two bundles parallel + scp to transfer the file, and parallel + ssh to get the result. But I can't figure out how to implement this with one bundle of parallel + ssh, simultaneously transfer the file and get the result.

parallel --tagstring {} -j 25 -q -a file_ip_addr_hosts.csv\  
sshpass -p pass ssh user@{}\  
'cat > file_for_send.sh && source file_for_send.sh && echo result' < file_for_send.sh

It is clear that the file is being created, but it is empty.
Everything works fine without using gnu parallel.

My task is to copy and execute a file on multiple hosts using parallel with one command.

7
  • Your not going to be able to copy a file to multiple targets with a single command, you can use the command to each target certainly, but I am not aware of a way to make multiple simultaneous SSH connections and transfer a file.
    – Ramhound
    Commented Jun 14 at 6:05
  • I don't understand why it doesn't work. In theory, parallel is just a wrapper for ssh, which allows you to run many instances of it. Commented Jun 14 at 6:16
  • Can you enable any sort of verbose logging?
    – Ramhound
    Commented Jun 14 at 6:41
  • < file_for_send.sh is attached to the stdin of parallel. Even if this stdin ultimately is inherited by ssh (and I don't know if it is), it's not that each ssh will reopen the file and read it from the beginning. Write a script that takes (at least) user@server as an argument and handles a single case of ssh inside, including the redirection from file_for_send.sh. Then let parallel run the script multiple times. Maybe there is a clever option for parallel to fork (tee-like) its stdin, but I don't know the tool good enough to tell. Commented Jun 14 at 6:42
  • Ssh logs do not report anything interesting, except for the absence of '< file_for_send.sh' . Parallel verbose, also reports absence from the team '< file_for_send.sh' It feels like he's ignoring her. Commented Jun 14 at 7:09

1 Answer 1

0

According to a good tradition, I post my own solution. As usual, afterwards, it often turns out to be not simple, but very simple:

parallel –tagstring {} -q -a file_ip_addr_hosts.csv\
sshpass -p pass ssh user@{1} $(<file_for_send.sh)

Of course, there are costs, but they are nonsense. In fact, we are passing the script file for execution, but line by line.
Parallel treats this file as one of the data sources.
Therefore, if, for example, you execute your script in a separate shell
(when you get root access su - root sh -c), we combine all commands through &&, otherwise if you create variables in the process, they will be lost.
I didn't find anything about it in the documentation, although I may have searched badly.
If this construction is allowed through jumpphost, then ‘$(<file_for_send.sh )’ we put it in quotation marks.
There is another point, when passing through jumphost, single quotes in the script (file_for_send.sh ) replace with

‘\'’

Maybe it will help someone.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .