RHCSA - Create Simple Shell Scripts: Processing Output of Shell Commands Within a Script

Processing the output of shell commands within a script itself is very useful. For this objective we will look at command substitution and piping command output.

Command Substitution

Command substitution allows you to execute a command and substitute its output as input into another command, or assignment to a variable for later use. It is denoted by the use of backticks `` or the dollar sign and parentheses $().

The use of parentheses is the preferred method in scripts as it is more readable, being able to clearly distinguish matching brackets.

Use command substitution so a scripts log file has a timestamp:

mkdir -p ~/scripts
cd ~/scripts

cat << 'EOF' > process_outputs_sub.sh 
#!/bin/bash

LOG="process_outputs_sub.$(date '+%Y%m%d%H%M%S').log"

echo "This script has executed" | tee $LOG

EOF

chmod u+x process_outputs_sub.sh

./process_outputs_sub.sh

The above will create a script that writes to a log file the message This script has executed. The log file will have a datestamp because it uses command substitution($(date '+%y%m%d%H%M%S')). We can check the script has ran correctly and produced the log file by running:

ls -l process_outputs_sub*log

Use command substitution to get a total line count of each log file in the /var/log directory:

mkdir -p ~/scripts
cd ~/scripts

cat << 'EOF' > process_outputs_sub2.sh 
#!/bin/bash

wc -l $(find /var/log/ -maxdepth 1 -type f -name "*.log") 2> /dev/null

EOF

chmod u+x process_outputs_sub2.sh

./process_outputs_sub2.sh

Here we need to count lines in the log files in the /var/log/ directory. The find command is run with the results substituted in its place for the wc word count command to run against.

Only the log files you have permission to read will be displayed.

Pipes

Pipes (|) allow you to take the output of one command and use it as input for another command. This is useful for chaining commands together and processing data through a series of transformations.

Use pipes to filter user input so the output contains letters only:

mkdir -p ~/scripts
cd ~/scripts

cat << 'EOF' > process_outputs_pipe.sh 
#!/bin/bash

INPUT="$*"

echo $INPUT | sed 's/[^a-zA-Z]//g'

EOF

chmod u+x process_outputs_pipe.sh

./process_outputs_pipe.sh 'abc123!"£def456$%^'

In this script the INPUT variable is set using $* to capture as a single string what the user provides as input. echo is used to output the INPUT variable, which then gets piped to the sed command, which then removes all characters but letters.

Chain multiple pipes together:

mkdir -p ~/scripts
cd ~/scripts

cat << 'EOF' > process_outputs_pipe2.sh 
#!/bin/bash

INPUT="$*"

echo $INPUT | sed 's/[^a-zA-Z]//g' | tr '[:lower:]' [:upper:]

EOF

chmod u+x process_outputs_pipe2.sh

./process_outputs_pipe2.sh 'abc123!"£def456$%^'

Building on the previous exercise, this time we are chaining multiple commands together, each working on the previous commands output.

Piping (streaming) output to a script for processing:

mkdir -p ~/scripts
cd ~/scripts

cat << 'EOF' > process_outputs_pipe3.sh 
#!/bin/bash

tr '[:lower:]' [:upper:]

EOF

chmod u+x process_outputs_pipe3.sh

echo 'abc DEF ghi JKL' | ./process_outputs_pipe3.sh

In this script the tr command is used to translate the piped data into uppercase letters. As the tr command is the first command in the script that data can be piped to, it will act as if the script itself is running echo 'abc DEF ghi JKL' | tr '[:lower:]' [:upper:]. Once the streamed data is used, it will not be available to susequent commands (unless stored to a file or variable).

Support DTV Linux

Click on each book below to review & buy on Amazon. As an Amazon Associate, I earn from qualifying purchases.

NordVPN ®: Elevate your online privacy and security. Grab our Special Offer to safeguard your data on public Wi-Fi and secure your devices. I may earn a commission on purchases made through this link.