Executing Commands with bash

This two-part article will explain how to launch programs in the bash shell, used with Linux and Unix operating systems. It is excerpted from chapter four of the bash Cookbook, Solutions and Examples for bash Users, written by Carl Albing, JP Vossen and Cameron Newham (O’Reilly, 2007; ISBN: 0596526784). Copyright © 2007 O’Reilly Media, Inc. All rights reserved. Used with permission from the publisher. Available from booksellers or direct from O’Reilly Media.

The main purpose of bash (or of any shell) is to allow you to interact with the computer’s operating system so that you can accomplish whatever you need to do. Usually that involves launching programs, so the shell takes the commands you type, determines from that input what programs need to be run, and launches them for you.

Let’s take a look at the basic mechanism for launching jobs and explore some of the features bash offers for launching programs in the foreground or the background, sequentially or in parallel, indicating whether programs succeeded and more.

4.1  Running Any Executable

Problem

You need to run a command on a Linux or Unix system.

Solution

Use bash and type the name of the command at the prompt.

  $ somepro g

Discussion

This seems rather simple, and in a way it is, but a lot goes on behind the scenes that you never see. What’s important to understand about bash is that its basic operation is to load and execute programs. All the rest is just window dressing to get ready to run programs. Sure there are shell variables and control statements for looping and if / then / else branching, and there are ways to control input and output, but they are all icing on the cake of program execution.

So where does it get the program to run?

bash will use a shell variable called $PATH to locate your executable. The $PATH vari able is a list of directories. The directories are separated by colons ( : ). bash will search in each of those directories for a file with the name that you specified. The order of the directories is important—bash looks at the order in which the directories are listed in the variable, and takes the first executable found.

  $ echo $PAT H
  /bin:/usr/bin:/usr/local/bin:.
  $

In the $PATH variable shown above, four directories are included. The last directory in that list is just a single dot (called the dot directory, or just dot), which represents the current directory. The dot is the name of the directory found within every directory on a Linux or Unix file system—wherever you are, that’s the directory to which dot refers. For example, when you copy a file from someplace to dot (i.e., cp  /other/place/file  . ), you are copying the file into the current directory. By having the dot directory listed in our path, bash will look for commands not just in those other directories, but also in the current directory (.).

Many people feel that putting dot on your $PATH is too great a security risk—someone could trick you and get you to run their own (malicious) version of a command in place of one that you were expecting. Now if dot were listed first, then someone else’s version of ls would supersede the normal ls command and you might unwit tingly run that command. Don’t believe us? Try this:

  $ bas h
  $ cd
  $ touch ls
  $ chmod 755 ls
  $ PATH=".:$PATH"
  $ ls
  $

Suddenly, the ls appears not to work in your home directory. You get no output. When you cd to some other location (e.g., cd  /tmp ), then ls will work, but not in your home directory. Why? Because in that directory there is an empty file called ls that is run (and does nothing—it’s empty) instead of the normal ls command located at /bin/ls. Since we started this example by running a new copy of bash, you can exit from this mess by exiting this subshell; but you might want to remove the bogus ls command first:

  $ cd
  $ rm ls
  $ exit
  $

Can you see the mischief potential of wandering into a strange directory with your path set to search the dot directory before anywhere else?

If you put dot as the last directory in your $PATH variable, at least you won’t be tricked that easily. Of course, if you leave it off altogether it is arguably even safer and you can still run commands in your local directory by typing a leading dot and slash character, as in:

  $ ./myscript

The choice is yours.

Never allow a dot or writable directories in root’s $PATH . For more, see Recipe 14.9, “Finding World-Writable Directories in Your $PATH” and Recipe 14.10, “Adding the Current Directory to the $PATH.”

Don’t forget to set the file’s permissions to execute permission before you invoke your script:

  $ chmod a+x ./myscript
  $ ./myscript

You only need to set the permissions once. Thereafter you can invoke the script as a command.

A common practice among some bash experts is to create a personal bin directory, analogous to the system directories /bin  and /usr/bin where executables are kept. In your personal bin you can put copies of your favorite shell scripts and other
custom ized or private commands. Then add your home directory to your $PATH , even to the front ( PATH=~/bin:$PATH ). That way, you can still have your own customized favorites without the security risk of running commands from strangers.

See Also

{mospagebreak title=4.2 Telling If a Command Succeeded or Not}

Problem

You need to know whether the command you ran succeeded.

Solution

The shell variable $? will be set with a non-zero value if the command fails—provided that the programmer who wrote that command or shell script followed the established convention:

  $ somecomman d
  it works…
  $ echo $?
  0
  $ badcommand
  it fails…
  $ echo $?
  1
  $

Discussion

The exit status of a command is kept in the shell variable referenced with $?. Its value can range from 0 to 255. When you write a shell script, it’s a good idea to have your script exit with a non-zero value if you encounter an error condition. (Just keep it below 255, or the numbers will wrap around.) You return an exit status with the exit statement (e.g., exit 1 or exit 0 ). But be aware that you only get one shot at reading the exit status:

  $ badcommand
  it fails…
  $ echo $?
  1
  $ echo $?
  0
  $

Why does the second time give us 0 as a result? Because the second time is reporting on the status of the immediately preceding echo command. The first time we typed echo $? it returned a 1 , which was the return value of badcommand. But the echo command itself succeeds, therefore the new, most-recent status is success (i.e., a 0 value). So you only get one chance to check it. Therefore, many shell scripts will immediately assign the status to another shell variable, as in:

  $ badcommand
  it fails…
  $ STAT=$?
  $ echo $STAT
  1
  $ echo $STAT
  1
  $

We can keep the value around in the variable $STAT and check its value later on.

Although we’re showing this in command-line examples, the real use of variables like $? comes in writing scripts. You can usually see if a command worked or not if you are watching it run on your screen. But in a script, the commands may be running unattended.

One of the great features of bash is that the scripting language is identical to com mands as you type them at a prompt in a terminal window. This makes it much easier to check out syntax and logic as you write your scripts.

The exit status is more often used in scripts, and often in if statements, to take different actions depending on the success or failure of a command. Here’s a simple example for now, but we will revisit this topic in future recipes:

  $ somecomman d
  …
  $ if (( $? )) ; then echo failed ; else echo OK; fi

See Also
  1. Recipe 4.5, "Deciding Whether a Command Succeeds"
  2. Recipe 4.8, "Displaying Error Messages When Failures Occur"

{mospagebreak title=4.3 Running Several Commands in Sequence}

Problem

You need to run several commands, but some take a while and you don’t want to wait for the last one to finish before issuing the next command.

Solution

There are three solutions to this problem, although the first is rather trivial: just keep typing. A Linux or Unix system is advanced enough to be able to let you type while it works on your previous commands, so you can simply keep typing one command after another.

Another rather simple solution is to type those commands into a file and then tell bash to execute the commands in the file—i.e., a simple shell script.

Assume that we want to run three commands: long, medium, and short, each of whose execution time is reflected in its name. We need to run them in that order, but don’t want to wait around for long to finish before starting the other commands. We could use a shell script (aka batch file). Here’s a primitive way to do that:

  $ cat > simple.script
  long
  medium
 
short
  ^D               # Ctrl-D, not visible
  $ bash ./simple.script

The third, and arguably best, solution is to run each command in sequence. If you want to run each program, regardless if the preceding ones fail, separate them with semicolons:

  $ long ; medium ; short

If you only want to run the next program if the preceding program worked, and all the programs correctly set exit codes, separate them with double-ampersands:

  $ long && medium && short

Discussion

The cat example was just a very primitive way to enter text into a file. We redirect the output from the command into the file named simple.script (for more on redirecting output, see Chapter 3). Better you should use a real editor, but such things are harder to show in examples like this. From now on, when we want to show a script, we’ll just either show the text as disembodied text not on a command line, or we will start the example with a command like cat filename to dump the contents of the file to the screen (rather than redirecting output from our typing into the file), and thus display it in the example.

The main point of this simple solution is to demonstrate that more than one command can be put on the bash command line. In the first case the second command isn’t run until the first command exits, and the third doesn’t execute until the second exits and so on, for as many commands as you have on the line. In the second case the second command isn’t run unless the first command succeeds, and the third doesn’t execute until the second succeeds and so on, for as many commands as you have on the line.

{mospagebreak title=4.4 Running Several Commands All at Once}

Problem

You need to run three commands, but they are independent of each other, and don’t need to wait for each other to complete.

Solution

You can run a command in the background by putting an ampersand (&) at the end of the command line. Thus, you could fire off all three jobs in rapid succession as follows:

  $ long &
 
[1] 4592
  $ medium &
  [2] 4593
  $ short
  $

Or better yet, you can do it all on one command line:

  $ long & medium & short
  [1] 4592
 
[2] 4593
  $

Discussion

When we run a command in the background (there really is no such place in Linux), all that really means is that we disconnect keyboard input from the command and the shell doesn’t wait for the command to complete before it gives another prompt and accepts more command input. Output from the job (unless we take explicit action to do otherwise) will still come to the screen, so all three jobs will be interspersing output to the screen.

The odd bits of numerical output are the job number in square brackets, followed by the process ID of the command that we just started in the background. In our example, job 1 (process 4592) is the long command, and job 2 (process 4593) is medium.

We didn’t put short into the background since we didn’t put an ampersand at the end of the line, so bash will wait for it to complete before giving us the shell prompt (the $ ).

The job number or process ID can be used to provide limited control over the job. You can kill the long job with kill %1 (since its job number was 1 ). Or you could specify the process number (e.g., kill 4592 ) with the same deadly results.

You can also use the job number to reconnect to a background job. Connect it back to the foreground with fg %1 . But if you only had one job running in the background, you wouldn’t even need the job number, just fg by itself.

If you start a job and then realize it will take longer to complete than you thought, you can pause it using Ctrl-Z, which will return you to a prompt. You can then type bg to un-pause the job so it will continue running in the background. This is basically adding a trailing & after the fact.

See Also

Chapter 2 on redirecting output

{mospagebreak title=4.5 Deciding Whether a Command Succeeds}

Problem

You need to run some commands, but you only want to run certain commands if certain other ones succeed. For example, you’d like to change directories (using the cd command) into a temporary directory and remove all the files. However, you don’t want to remove any files if the cd fails (e.g., if permissions don’t allow you into the directory, or if you spell the directory name wrong).

Solution

We can use the exit status ($?) of the cd command in combination with an if statement to do the rm only if the cd was successful.

  cd mytm p
  if (( $? )); then rm * ; fi

Discussion

Obviously, you wouldn’t need to do this if you were typing the commands by hand. You would see any error messages from the cd command, and thus you wouldn’t type the rm command. But scripting is another matter, and this test is very well worth doing to make sure that you don’t accidentally erase all the files in the directory where you are running.

Let’s say you ran that script from the wrong directory, one that didn’t have a subdirectory named mytmp. When it runs, the cd would fail, so the current directory remains unchanged. Without the if check (the cd having failed) the script would just continue on to the next statement. Running the rm * would remove all the files in your current directory. Ouch. The if is worth it.

So how does $? get its value? It is the exit code of the command. For C Language programmers, you’ll recognize this as the value of the argument supplied to the exit() function; e.g., exit(4); would return a 4. For the shell, zero is considered success and a non-zero value means failure.

If you’re writing bash scripts, you’ll want to be sure that your bash scripts explicitly set return values, so that $? is set properly from your script. If you don’t, the value set will be the value of the last command run, which you may not want as your result.

See Also
  • Recipe 4.2, "Telling If a Command Succeeded or Not"
  • Recipe 4.6, "Using Fewer if Statements"

Please check back next week for the conclusion to this article.

[gp-comments width="770" linklove="off" ]

chat