Fernando Ipar » shell

Piping data to multiple processes

fernando — Thu, 10 Mar 2011 05:00:06 +0000

Here’s a simple shell script to stream data to multiple processes. It has many applications, but the reason I wrote it is to stream the same data to multiple netcat processes on remote machines.

Here’s the code:

#!/bin/bash

usage()
{
cat <&2

usage : multi-fifo target0 [target1 [target2 [...]]]

Where each targetN is a program you want to send the input multi-fifo receives

EOF

}

[ $# -eq 0 ] && usage && exit 1

i=0
pipeline="tee /dev/null"
while [ -n "$1" ]; do
target=$1
fifo=/tmp/multi-fifo-$$.$i
mkfifo $fifo
eval "$target < $fifo" &
pipeline="$pipeline | tee $fifo"
i=$((i+1))
shift
done

eval "cat | $pipeline"

j=0
while [ $j -lt $i ]; do
rm -f /tmp/multi-fifo-$$.$j
j=$((j+1))
done

Usage is pretty straightforward:

tar cjvf - . | ./multi-fifo "nc host1 9999" "nc host2 9999" >/dev/null

This way, the script will create a fifo for each netcat client, then send it’s stdin to a pipeline that writes to both fifos using tee. Useful for example if you want to stream a backup to multiple destination servers for cloning.

Using the ENUM data type to increase performance While going through the DATA TYPES section of the Certification...
Generating data with dbmonster In my last post I included some sample data...
Running commands from the shell with a timeout (pt 2) Here’s an improved version of the safecmd script. This one...

Related posts brought to you by Yet Another Related Posts Plugin.

Running commands from the shell with a timeout (pt 2)

fernando — Mon, 12 Jan 2009 22:45:33 +0000

Here’s an improved version of the safecmd script. This one doesn’t always wait for $timeout seconds even if the command you’re running exits successfully. In that case, it kills the monitor script and ends at the proper time.

Here’s the code:

#!/bin/bash 

timeout=$1
command=$2
shift 2

check_pid_name()
{
        command=$1
        childpid=$2
        [ -d /proc/$childpid ] || return 1
        [ $(grep -c $command /proc/$childpid/cmdline 2>/dev/null) -gt 0 ] && return 0 || return 1
}

[ -z "$command" ] && echo "Usage: safecmd   [args]">&2 && exit 1

safe_run()
{
        command=$1
        shift
        $command $*
        kill $$
}

safe_run $command $* &
childpid=$!
sleep $timeout 

check_pid_name $command $childpid && {
        kill $childpid
        sleep 0.1
        check_pid_name $command $childpid && kill -9 $childpid
        echo "$command $* timed out"
}

You can try it how like this, for example:

./safecmd 2 sleep 4

which will output

./safecmd.1: line 34: 11739 Terminated               safe_run $command $*
sleep 4 timed out

or ./safecmd 2 sleep 1

which will output

Terminated

Indicating that sleep 1 finished successfully

This is still missing something. You can’t test for the success of the command you pass safecmd, if it exits successfully you’ll see the same return code than if it would have exited with an error.

In order to improve this, as far as I can tell, you have to use two scripts, one just as the wrapper (to use one separate script instead of the function safe_run()). That’s the way this is handled in Highbase, and I’ll post a full example in my next post on this subject.

Running commands from the shell with a timeout Sometimes, in a shell script, you need to run a...
Generating random salts from bash From the ‘just because it can be done’ column, here...

Related posts brought to you by Yet Another Related Posts Plugin.

Running commands from the shell with a timeout

fernando — Sat, 10 Jan 2009 02:21:49 +0000

Sometimes, in a shell script, you need to run a command that might go on forever (or an amount of time long enough to be called forever) due to poor coding of the software and wrong external conditions (network down or overloaded, overloaded system, a changed private key, etc).

In order to avoid this situations, here’s a small shell script that will execute any command with a guaranteed timeout:

#!/bin/bash

timeout=$1
command=$2

check_pid_name()
{
        command=$1
        childpid=$2
        [ -d /proc/$childpid ] || return 1
        [ $(grep -c $command /proc/$childpid/cmdline 2>/dev/null) -gt 0 ] && return 0 || return 1
}

[ -z "$command" ] && echo "I need a command to run">&2 && exit 1

[ $# -eq 2 ] && shift || shift 2

$command $* &
childpid=$!

sleep $timeout 

check_pid_name $command $childpid && {
        kill $childpid
        sleep 0.1
        check_pid_name $command $childpid && kill -9 $childpid
        echo "$command $* timed out"
}

As you can see, it’s quite simple. You just have to start the desired command in the background, sleep for the timeout, and then kill the process if it’s still running.

Yes, this script has a defect, all the commands will wait $timeout to run, but that’s easy to solve. I’ll post an improved (but slightly more complex) version in my next post.

Running commands from the shell with a timeout (pt 2) Here’s an improved version of the safecmd script. This one...
Generating random salts from bash From the ‘just because it can be done’ column, here...
Piping data to multiple processes Here’s a simple shell script to stream data to multiple...

Related posts brought to you by Yet Another Related Posts Plugin.