Shell Basics

I have been using Linux and its command shell for many years now. And like any other Linux veteran I don't think twice about how to get a script working which I find on the internet. However, whenever I publish one of my scripts here on this web site I've got to find a trade off between being very verbose or being very brief about how to enter and run a script. Being very verbose is boring, both for me to write as for an advanced Linux user to read. Being very brief might scare some new Linux users off, or it may raise many questions which need to be answered.

That's why I've decided to write this page. It is intended for the (hopefully many) new Linux users which are attracted by the Raspberry Pi.

Before We Begin

Let me start with some very trivial things you need to know before we really dive into entering and running scripts. Most people who are new to Linux have a Windows background, with hopefully at least a little knowledge about DOS. So let's start by examining the most import differences between Linux and Windows.

File names are case sensitive

First of all, file names in Linux are case sensitive. This means that the file named example.txt is a different file than the one called Example.txt. They both can live in the same directory simultaneously, which would have been impossible in Windows.

Directory separators

In Linux a directory path is separated by the forward slash /, in Windows the backward slash \ is used for that task. In Linux the backward slash has a totally different meaning. It is used to "Escape" the next character. Escaping a character is a way to tell Linux to treat the next character literally, instead of treating it as it usually would. I'm sure Google can explain the process of Escaping a character with the backward slash further to you.

There's only one root to the tree

Linux has only one root directory. This means that any file in the system can be found by following a specific path from this root directory. In Windows there are a total of 26 possible root directories, one on each letter of the alphabet.
In Linux the root directory (the first directory of the directory tree) is represented by the forward slash /.

In Linux multiple file system, like CD ROM or USB thumb drives, are mounted on a so called mount point. A mount point is basically an empty directory which will be used to hold the mounted file system. A CD ROM for instance could be mounted, and thus accessed, from the directory /media/cdrom. When the CD is not mounted the directory /media/cdrom will be empty, or may not even exist.

Hidden files

A file or directory name starting with a dot . is a so called hidden file or directory. The file exists, but can't be seen when you perform a normal directory listing with the ls command. You can make the file or directory visible with the -a flag, like ls -a .
It's not that hidden files or directories contain secrets. They usually contain important settings, which usually are not interesting enough when you want to search for a particular file.

Although you can't see hidden files in a normal directory listing, you don't have to do anything special to access them. Editing the file .bashrc can simply be done by entereing the command nano .bashrc.

This directory and its parent

Each directory has two special entries in it (except the root directory, which has only one special entry). A single dot, meaning "this directory". And two dots .. meaning the parent directory (one directory back towards the root).
So if you want to change the directory back to the parent directory type: cd .. If you want to go two directories back you can type cd ../..

Multi user

Linux is a true multi user platform, which is completely different from the approach Windows uses. This means that in Linux multiple users can login to the machine at the same time. In the Raspberry Pi you have probably already met two of those users: pi and root (A.K.A. the super user). Linux knows many more users. Most of the background processes on the machine are owned by special system users. That way those processes can only access files which are owned by that particular system user. Should such a process go crazy, because of a bug or a malicious break in, it can only destroy its own files. It won't be able to destroy files owned by other users.

Each real user (not the system users I've mentioned above) has its own home directory. All user's home directories can be found in the directory /home. So the home directory of the user pi can be found in the directory /home/pi. The user can do whatever he wants in that directory. He can even delete all files in that directory, something he can't do in home directories of other users.
There's only one exception. The root user has his home directory in /root, not in /home/root.
A user's home directory path can be referenced by a tilde symbol ~/ So for user pi the paths /home/pi and ~/ both point to the same directory, pi's home directory.

The user named root is the super user. He can do everything, he can even delete important files, which could destroy the system. Therefore most Linux veterans will discourage you to be the root user all the time.
As a normal user you can perform root's tasks by preceding the command you want to execute by the sudo command. So if you want to edit a system configuration file, which only root is allowed to do, you would for instance type: sudo nano /etc/hosts .

Passwords

Don't be surprised when nothing appears on the screen whenever you type a password in the terminal. This is a security measure. Someone looking over your shoulder would not be able to see the length of your password.

Access rights

To keep things safe and private a set of access rights are defined. In short each file has separate rights for its owner, for a group of users and for the rest of the system. B.T.W. the root user can bypass all rights, so he can read any file on the system, no matter who owns it.

Let me explain users and groups first. The user who creates a file automatically becomes the owner of that file. For him the user access rights apply. Each user belongs to one or more groups (type the id command to see the groups you belong to). You are always part of one main group (the one mentioned after gid in the output of the id command). Any new file you create automatically belongs to that group. There are commands to change the owner and/or group a file should belong to (chown and chgrp).

And now about the rights. Each user, each group and the rest of the system have their own set of access rights on each file. There are 3 types of access rights, the right to read the file, the right to write to the file and the right to execute a file. Normally, on any new file, the owner gets read and write access, the group gets read and write access while the rest of the system gets only read rights.
Please note that you will never automatically get execute rights if you create a new file. That's one of the many defenses in Linux against running malicious code. You can see the individual rights of each file by using the -l flag for the ls command, like ls -l . The beginning of each line output by this command looks something like: -rwxrw-r-- Each position is a separate flag. If the flag is set, it is shown as a letter. If the flag is not set (not active) it is shown as a - place holder. You read the previous example as:

  • The first flag (the left dash) can be many things. If it's a dash, it means it is a normal file. If it is the letter d, it means it is a directory. If it is the letter l, it means it is a symbolic link (like a short cut in Windows).
  • Then the next 3 flags (rwx) show that the user (the owner of the file) has total rights. He can read the file, he can write the file and he can execute the file.
  • The next 3 flags (rw-) show that the group to which the file belongs can only read and write the file. They have no right to execute the file.
  • And the final 3 flags (r--) show that the rest of the system only have the right to read the file. They are not allowed to write to it or execute it.

You can use the chmod command to change the access rights.

The third and fourth column in the output of the ls -l command show the user and group a file belongs to. You can change the owner or group with the chown command. Only the root user can change the owner of a file, while a normal user can change the group of a file from the set of groups he is a member of.

The shell

And finally I would like to spend a few words about the shell. The shell is a wrapper around the system which enables you to type commands and read the results of those commands. Linux knows many different shells. Not so long ago a shell by the name of sh was the standard on all Linux Systems. That was the shell to use when you wanted to write cross platform scripts, because you could be very certain that sh was available on most other machines.

However the sh shell has its limitations. Many other shells were invented with new and more advanced features. Nowadays the shell named bash has replaced sh as the standard shell. It is safe to write your scripts in bash where you can be pretty sure that the shell exists on other systems, or at least can be installed on them.
Bash is the default shell on your Raspberry Pi.

I think it's good to known the name of the shell, because you can use it in Google to find more details about its capabilities.

Running Scripts In Linux

Running a script, or indeed any kind of program, from the command line in Linux is usually a matter of typing the name of the script or program at the command prompt. This name may or may not be followed by some options or parameters. And to be complete, at the end of line you hit the Enter key.
The next three lines for instance show you the directory listing of the current directory, the long version of the current working directory and the contents of the /etc directory respectively. 

ls
ls -l
ls /etc

The first line was a command on its own. The second line was a command with an option behind it. The third line was a command with a parameter behind it. Simply type these lines, one by one, on the command line of your Raspberry Pi and see the differences.

BTW: If you want to find out more about a particular command you can type man commandname. This will show you the manual page of that particular command, if it exists. Man pages show you how to use that command, what options you can use and what other parameters it expects. Linux is famous for its comprehensive man page system. Most of the official commands come with such a man page.

Every command you type on the command line is a program or script, stored on the computer's hard disk (or SD card in our case). Well there are a few exceptions, like cd and exit. These are so called built in commands, which don't require an external program. But for now it suffices to say that every command is an executable file, somewhere on the machine's disk.
If you want to know what program is executed by a specific command you can use the which command. 

which man

/usr/bin/man

But where does the system find these commands? Well, type the next command and you'll see all the directories where the system will look to find commands. 

echo $PATH

$PATH is a so called environment variable. It's name is in all capitals, which is important for Linux. The list you'll see after typing the above command tells you where the system will look for commands that you type. Every directory in the path is separated from the next by a colon. First the system will look in the left most directory for the command you have typed. If it can't find it there it will continue to the next directory. It continues all the way to the end of the list, until it finds your command.
Once the command is found, it is executed. Bash will tell you if it can't find the command in the path. If the command doesn't exist you may have made a typing error. Or the particular command is not installed on the current system. In that case you are told how to install it.
If a particular command exists in more than one location, only the first program found will be executed. You'll have to provide the full path to the command if you want to execute the other command.

All is well and good for system commands. But what about programs we write our selves? There are three ways to execute them.

  • Include the path to your own program when you try to run it.
  • Create a directory called bin in your home director and put the script/program in that directory.
  • Put your program or script in one of the path directories.

Including the path to your program or script is easy. The path may be absolute or relative. In most cases you want to execute a script from your current directory location. So your script can be run by typing ./yourscript . Remember that . is the same as your current working directory, just like .. was the parent of the current directory. And / is the directory separator.

On Raspbian (and many other Linux distributions) a directory called bin in your home directory will automatically be added to the execution path if it exists. So all you need to do is create a directory called mkdir ~/bin in your home directory (usually /home/pi/bin), and log out and log back in. Then you can put your scripts into this bin directory and your shell will find and execute them if you just type the script name, provided the execute flag of the script is set of course.

And finally for the last option, you can put your scripts and self made programs in a system directory (usually /usr/bin. All users of your RPi can then benefit from the scritps. You usually would only do this once the script is ready, because only the root user can write to this directory.

OK, the shell can now find your script, or at least we know how to explicitly tell it where to find it. But it still won't execute the script without some further steps. For one thing the file must be made executable. We do this with the command chmod u+x scriptname. The +x means we want to switch on the execute flag, while the u means we want to do that for the user (or owner) of the file. chmod +x scriptname would swich on the execute flag for all three types (User, Group and rest of the system).

The script must have the correct file header. You can name your script anything you like, with or without an extension. The system will look at the file header to find out what to do with it. For instance a bash shell script should start with the following header (first line of the script): 

#! /bin/bash

This means that the script is to be executed (interpreted in this case) by the program /bin/bash. Another example would be: 

#! /usr/bin/python

This script would be the first line of a python script.

A compiler will take care of providing the correct header for compiled programs.

Text Editor

If you want to enter or edit a script file you'll need a text editor. This is something entirely different from a word processor! A word processor can change font styles and colours and other properties of the text to make it print nicely. A text editor lacks all these features. Its only purpose is to compose flat text, which in this case makes up the program. It doesn't have to look pretty, it simply must contain all the commands you need to get the job done.

Linux comes with dozens of text editors. Which one is best is just a matter of own preferences. For years the editor called vi has been the default editor in the Linux world. No doubt it is a powerful editor, but it is definitely not the easiest one to learn. For that reason the editor called nano has slowly taken over the role of being the default editor in Linux. This means that you can be pretty sure that nano is installed on most of the Linux distributions, just like vi has always been. It doesn't mean that you'll have to use nano though, you can use any editor you like. It's your Raspberry Pi, so you can install whatever you like.

Here's a small list of editors you can use in the terminal of your Raspberry Pi. The list is by no means complete. So please don't shoot me if I have skipped your favourite editor.

  • vi
    Available on just about all Linux distributions. Has a very steep learning curve though.
  • vim
    This is an improved version of vi. I have no experience with this editor.
  • nano
    The default editor on the Raspberry Pi and most other Linux distributions.
  • pico
    The predecessor of nano, which is part of the pine mail program.
  • emacs
    State of the art text editor.
  • jed
    A sort of emacs clone. Basically uses the same command set.
  • joe
    Just another high end editor.
  • leafpad
    A GUI based text editor, similar to Notepad on Windows.

As a general rule of thumb though you can say that the more features an editor has, the steeper the learning curve will be. So I would suggest you stick to nano for a while, until you feel more confident to take programming a step further.
The best feature in nano is the fact that it shows you a summery of the most often used commands on the screen, so you don't have to memorize them.