The abridged guide to shell scripting and the command line¶

Finding out more about a command¶

Use the man program to read the documentation for any given command/program. To learn more about cat, for example:

$ man cat

Reading/writing/input/output¶

How to download a URL

Use the curl program, which is basically ubiquitous across all command-line systems:

$ curl https://www.nytimes.com

How to write to a file

Use the > operator to signify that you want to send the output of one program into a file at the given filename. Whatever is currently at that filename gets wiped out:

$ curl https://www.nytimes.com > nytimesfrontpage.html

$ echo 'oops' > nytimesfrontpage.html

How to print the contents of a file to screen (standard output)

For text files, use cat:

$ cat nytimesfrontpage.html

How to direct the output of one program into another program

That’s the pipe, a concept so integral to the command-line that you just have to memorize it:

$ curl https://www.nytimes.com | ack 'Trump'

File and directory navigation¶

What directory am I currently in?

Use pwd (print working directory) to print out the full path of the current working directory. Any operation you do, such as creating new files/directories, are relative to the current working directory.

$ pwd

In general, you should never do anything that creates, accesses, or destroys a file/directory without making sure you know exactly where you are.

What files and subdirectories exist in the current working directory?

Run the ls command (“list directory contents”) without any arguments:

$ ls

How do I move up to the parent directory?

The double dot character sequence is interpreted as “the parent directory”.

For example, if you start out in your home directory, and then you cd (change directory) into your Downloads subdirectory, which itself has a movies subdirectory:

$ cd Downloads/movies

To get back to Downloads, i.e. the parent directory of movies:

$ cd ..

How do I jump back to my “home” directory?

You can either call cd without any arguments:

$ cd

Or use the ~ (tilde), which is interpreted as a shortcut for the home directory:

$ cd ~

What computer am I on? Who am I?

Seems silly, but you’re going to frequently find yourself logging/tunneling into remote computers to do things like launch a web application or an annoying Twitter bot. Like Leonardo DiCaprio in “Inception”, you might forget what system you’re actually on.

Use whoami and hostname, respectively, to print out your username and the name of the computer.

Tab or Die¶

The Tab key is used for autocompletion of programs and filenames that exist on the system. That is its function in almost every programming environment, whether its the Atom or Sublime Text editor, the Unix shell of the 80s or of today, or the Python shell.

If you’re programming, and you aren’t hitting Tab several times a minute, you are probably doing something wrong, or about to do something catastrophic.

Relying on the Tab key is a realization that our human minds cannot be trusted to remember exactly the correct spelling of archaic program names, nor our fingers to type without making typos, nor our eyes to reliably differentiate a l from 1 or I.

Here’s a nice friendly explainer by How-To Geek: Use Tab Completion to Type Commands Faster on Any Operating System

Use Ctrl-C to break out of confusing situations¶

Sometimes you’ll accidentally run a program that executes, oh, an infinite number of times. Like the yes program:

$ yes yes

Hit Ctrl-C on your keyboard to break execution of a program, or non-responsiveness from the interpreter. This also works in the Python shell.

Use the up-arrow to cycle through past commands¶

Running the history command will show you all the commands that you’ve recently run, in sequential order. But if you just need to re-run the previous command with a minor alteration, hit the Up arrow and the command will be fully written out at the prompt, ready for you to hit Enter.

Use Ctrl-A and Ctrl-E to jump to the beginning and end of a line¶

The command line, being all-text, doesn’t have the convenience of mouse-clicking to move the cursor, like we’re used to in a modern word processor or text editor. While you could just hold down the left or right arrow when you need to fix something at the beginning or end of a line, you should memorize using Ctrl-A to jump to the beginning of a line , or Ctrl-E, to jump to the end.

I think “A”, as in, the beginning of the alphabet. And “E”, as in, “End”.

Use Ctrl-L to clear the screen¶

Command-line programs spit a lot of text on to the screen. And usually, that output text can be a distraction. Use Ctrl-L to start with a blank screen.

There is no undo¶

The following command will direct the text string, hello world, into the file named LIFESAVING_INFO.docx:

$ echo 'hello world' > LIFESAVING_INFO.docx

What if LIFESAVING_INFO.docx already exists and actually lives up to its name? Too bad, it now contains hello world, and there is no undelete or undo.

The following command uses cp to copy the contents of one file to another filename:

$ cp dumbfile.txt super-valuable-file.xlsx

Again, makes no difference if the destination file already exists – cp wipes out what was there.

Use single quotes to enclose arguments and values¶

Basically, if something is not a command – e.g. curl, echo, ls – then you probably want to wrap it in single-quotes. In the example below, the argument to the curl program – http://www.example.com is enclosed in single-quotes:

$ curl 'http://www.example.com'

To be fair, http://www.example.com is considered “safe”, in that it doesn’t have any special characters that will be interpreted/executed by the shell. However, the following URL, which is a valid URL that generates a Google static map, has special characters such as the ampersand which will not be read in as literal text by the standard Unix-like shell:

https://maps.googleapis.com/maps/api/staticmap?size=800x500&zoom=14&center=Stanford+CA

In other words, do not do this:

$ curl https://maps.googleapis.com/maps/api/staticmap?size=800x500&zoom=14&center=Stanford+CA > map.jpg

Enclose that URL argument in single quotes. Might as well enclose that simple destination filename too:

$ curl 'https://maps.googleapis.com/maps/api/staticmap?size=800x500&zoom=14&center=Stanford+CA' > 'map.jpg'

$ echo "$987,293"

$ echo $987,293

$ echo '$987,293'

Use the backslash at the end-of-a-line to continue a long command¶

The shell interprets each newline character as the end of a command. You notice this when, every time you hit Enter, the command you typed in gets executed.

So commands are read in, one line at a time. But for aesthetic and readability concerns, sometimes we need to break up a single command into separate lines.

To do this, put a backslash at the end of the line. Do not add any space after the backslash.

In the following example, the curl command has some verbose options (e.g. --silent) and the URL to download is very long.

The backslash is used to split the single command into two lines for easier reading.

$ curl --silent --output destination.file \
    http://somereally.long.weburl/thatis/too/long/for/easy/readability.html

Use the backslash to split up a pipe sequence¶

Note that if you end a line with a pipe character, the shell will anticipate that your sequence is not quite done and won’t execute until you type a line without a pipe at the end:

$ curl http://www.example.com |
    ack '\w+' |
    sort |
    uniq -c > wordcount.txt

However, I recommend that you use a backslash to signify continuation of a long pipe sequence. It’s interpreted the same by the shell, so my recommendation is purely a stylistic one that is easier to read and may prevent mistakes later on. Remember, you will spend far more time reading over your own code than actually writing it:

$ curl http://www.example.com \
    | ack '\w+' \
    | sort \
    uniq -c > wordcount.txt