Last meeting we setup the Raspberry Pi as a headless server. We discussed some of the commands needed to maintain the system.
Now we are going to discuss some of the background programs and configuration used by the operating system.
This will be a wide ranging talk about Linux on the Raspberry Pi.
At the end of the talk in December we left off the following topics.
System Start-up tasks
Updating the software
Installing programs not in the repositories
So tonight we are going to try to tackle these subjects.
When a computer starts up it goes through a series of steps before it is ready to execute user’s commands.
Taken from The boot sequence
Stage 1 boot is in the on-chip ROM. Loads Stage 2 in the L2 cache
Stage 2 is bootcode.bin. Enables SDRAM and loads Stage 3
Stage 3 is loader.bin. It knows about the .elf format and loads start.elf
start.elf loads kernel.img. It then also reads config.txt, cmdline.txt and bcm2835.dtb If the dtb file exists, it is loaded at 0×100 & kernel @ 0×8000 If disable_commandline_tags is set it loads kernel @ 0×0 Otherwise it loads kernel @ 0×8000 and put ATAGS at 0×100
kernel.img is then run on the ARM.
Everything is run on the GPU until kernel.img is loaded on the ARM.
The config.txt file is documented in Config.txt
You can consult a previous talk Raspberry Pi Boot Process for a comparison of the Linux boot process and that on the Raspberry Pi.
Once the Linux kernel is loaded and starts loading the required modules, the system starts the first application, systemd.
Systemd is the program that will start the other daemons used by the system. This initialization program is part of the Linux kernel. It uses configuration files stored in /etc/systemd/system.
To list all the services running under systemd use the command:
systemctl | less
To see a list of all services and their status use the command:
The Linux system contains a schedule program called cron. This daemon wakes up once per minute and checks a series of task lists looking for apps to run. The data is stored for both the system and the individual users.
The system entries are stored in the folders and files:
file: /etc/crontab directory: /etc/cron.d: directory: /etc/cron.daily: directory: /etc/cron.hourly: directory: /etc/cron.monthly: directory: /etc/cron.weekly:
The user entries are stored in the user’s home directory under the name crontab.
The format of these files is: (Lines beginning with # are comments.)
# * * * * * command to execute # ┬ ┬ ┬ ┬ ┬ # │ │ │ │ │ # │ │ │ │ │ # │ │ │ │ └───── day of week (0 - 7) (0 to 6 are Sunday to Saturday, or use names; 7 is Sunday, the same as 0) # │ │ │ └────────── month (1 - 12) # │ │ └─────────────── day of month (1 - 31) # │ └──────────────────── hour (0 - 23) # └───────────────────────── min (0 - 59)
To list the contents of your crontab use the command crontab -l. Or if you want to edit the contents use the command crontab -e.
One useful addition to the Raspberry Pi is a we server. You can check if you have one installed by going to
http://localhost or http://<ip address>
You should see a page like
If you do not get a web page you will need to install the Apache2 package using the command:
sudo apt install apache2
I find it useful to also install a web tool for scripting called php. You can use the following command to install it:
sudo apt install php
Once this is installed you should create a web page nano /var/www/html/phpinfo.php like this:
<?php phpinfo (); ?>
Now you can look at the new page with:
http://localhost/phpinfo.php or http://<ip address>/phpinfo.php
To get some ideas about what you can use a web server for have a look at the talk Web interface for Raspberry Pi Apps
When ever you use any Linux system you should be logged in as a regular user. This is security issue you do not damage or corrupt the system. But sometime you need to be the root user to perform an action. Typically you use the sudo application.
When you want to run a command as root, prefix the command with sudo. Here is an example
pi@raspberrypi:~ $ ls /etc/ssl/private ls: cannot open directory '/etc/ssl/private': Permission denied pi@raspberrypi:~ $ sudo ls /etc/ssl/private ssl-cert-snakeoil.key
If you wanted to stay as root user for a while you can use the command sudo -i to switch to the root user.
Caution: only stay as root user when you are performing system tasks, like configuration or updating software. Otherwise you will be bypassing one of the best security features of Linux.
The command line tool for maintaining a Raspberry Pi is apt (Advanced Packaging Tool). It is normally only used by the root use for installing, removing, and updating the software.
When you want to make sure your package list is up to date you use the command:
apt update # as root user or sudo apt update as a normal user.
This will query the package repositories and download the current list of packages.
To update a system use the commands:
apt update apt dist-upgrade
This is update the package repository lists, and then look for new versions of your current list of packages. When it finishes presenting you a list of packages to update it will ask you if you want to continue. A typical session will look something like this:
root@raspberrypi:~# apt dist-upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following packages will be upgraded: ffmpeg libarchive13 libav-tools libavcodec57 libavdevice57 libavfilter6 libavformat57 libavresample3 libavutil55 libobrender32v5 libobt2v5 libpam-systemd libpostproc54 libsdl1.2debian libssl1.0.2 libswresample2 libswscale4 libsystemd0 libudev1 libvlc-bin libvlc5 libvlccore9 libzmq5 lxappearance-obconf obconf openbox rpi-chromium-mods systemd systemd-sysv udev vlc vlc-bin vlc-data vlc-l10n vlc-plugin-base vlc-plugin-notify vlc-plugin-qt vlc-plugin-samba vlc-plugin-skins2 vlc-plugin-video-output vlc-plugin-video-splitter vlc-plugin-visualization 42 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Need to get 35.2 MB of archives. After this operation, 462 kB disk space will be freed. Do you want to continue? [Y/n]
Notice that it tells you how many packages will be upgraded, installed, removed, or not upgraded. Additionally it will tell you the disk usage needed. If you think this looks OK answer with y to continue.
You can search for packages with either apt or apt-cache. Apt is the newer version. Here is an example:
root@raspberrypi:~# apt search dwww Sorting... Done Full Text Search... Done doc-base/stable 0.10.7 all utilities to manage online documentation dwww/stable 1.13.3 armhf Read all on-line documentation with a WWW browser epydoc-doc/stable 3.0.1+dfsg-14 all tool for documenting Python modules (documentation) gri-html-doc/stable 2.12.23-10 all HTML manual for gri, a language for scientific graphics synaptic/stable 0.84.2+rpi1 armhf Graphical package manager
Or using the older version of the program apt-cache.
root@raspberrypi:~# apt-cache search dwww doc-base - utilities to manage online documentation dwww - Read all on-line documentation with a WWW browser epydoc-doc - tool for documenting Python modules (documentation) gri-html-doc - HTML manual for gri, a language for scientific graphics synaptic - Graphical package manager
As you can see the apt gives more information, but apt-cache works.
To install a program use the install option like so:
root@raspberrypi:~# apt install dwww Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: dctrl-tools dlocate doc-base info2www libfile-ncopy-perl libmime-types-perl libuuid-perl libyaml-tiny-perl swish++ Suggested packages: debtags rarian-compat doc-debian dpkg-www libmojolicious-perl xpdf-utils antiword The following NEW packages will be installed: dctrl-tools dlocate doc-base dwww info2www libfile-ncopy-perl libmime-types-perl libuuid-perl libyaml-tiny-perl swish++ 0 upgraded, 10 newly installed, 0 to remove and 0 not upgraded. Need to get 723 kB of archives. After this operation, 2,221 kB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://raspbian-us.ngc292.space/raspbian stretch/main armhf dctrl-tools armhf 2.24-2 [94.4 kB] Get:2 http://raspbian-us.ngc292.space/raspbian stretch/main armhf dlocate all 1.07+nmu1 [23.9 kB] Get:3 http://raspbian-us.ngc292.space/raspbian stretch/main armhf libuuid-perl armhf 0.27-1 [17.6 kB] Get:4 http://raspbian-us.ngc292.space/raspbian stretch/main armhf libyaml-tiny-perl all 1.70-1 [32.0 kB] Get:5 http://raspbian-us.ngc292.space/raspbian stretch/main armhf doc-base all 0.10.7 [100 kB] Get:6 http://raspbian-us.ngc292.space/raspbian stretch/main armhf libfile-ncopy-perl all 0.36-1 [14.3 kB] Get:7 http://raspbian-us.ngc292.space/raspbian stretch/main armhf libmime-types-perl all 2.13-1 [61.7 kB] Get:8 http://raspbian-us.ngc292.space/raspbian stretch/main armhf dwww armhf 1.13.3 [123 kB] Get:9 http://raspbian-us.ngc292.space/raspbian stretch/main armhf info2www all 220.127.116.11-24 [22.9 kB] Get:10 http://raspbian-us.ngc292.space/raspbian stretch/main armhf swish++ armhf 6.1.5-4 [233 kB] Fetched 723 kB in 4s (172 kB/s) Preconfiguring packages ... Selecting previously unselected package dctrl-tools. (Reading database ... 134811 files and directories currently installed.) Preparing to unpack .../0-dctrl-tools_2.24-2_armhf.deb ... Unpacking dctrl-tools (2.24-2) ... Selecting previously unselected package dlocate. Preparing to unpack .../1-dlocate_1.07+nmu1_all.deb ... Unpacking dlocate (1.07+nmu1) ... Selecting previously unselected package libuuid-perl. Preparing to unpack .../2-libuuid-perl_0.27-1_armhf.deb ... Unpacking libuuid-perl (0.27-1) ... Selecting previously unselected package libyaml-tiny-perl. Preparing to unpack .../3-libyaml-tiny-perl_1.70-1_all.deb ... Unpacking libyaml-tiny-perl (1.70-1) ... Selecting previously unselected package doc-base. Preparing to unpack .../4-doc-base_0.10.7_all.deb ... Unpacking doc-base (0.10.7) ... Selecting previously unselected package libfile-ncopy-perl. Preparing to unpack .../5-libfile-ncopy-perl_0.36-1_all.deb ... Unpacking libfile-ncopy-perl (0.36-1) ... Selecting previously unselected package libmime-types-perl. Preparing to unpack .../6-libmime-types-perl_2.13-1_all.deb ... Unpacking libmime-types-perl (2.13-1) ... Selecting previously unselected package dwww. Preparing to unpack .../7-dwww_1.13.3_armhf.deb ... Unpacking dwww (1.13.3) ... Selecting previously unselected package info2www. Preparing to unpack .../8-info2www_18.104.22.168-24_all.deb ... Unpacking info2www (22.214.171.124-24) ... Selecting previously unselected package swish++. Preparing to unpack .../9-swish++_6.1.5-4_armhf.deb ... Unpacking swish++ (6.1.5-4) ... Processing triggers for mime-support (3.60) ... Processing triggers for desktop-file-utils (0.23-1) ... Setting up dctrl-tools (2.24-2) ... Setting up libyaml-tiny-perl (1.70-1) ... Setting up libmime-types-perl (2.13-1) ... Setting up libuuid-perl (0.27-1) ... Setting up doc-base (0.10.7) ... Registering 31 doc-base files... Registering documents with dwww... Setting up info2www (126.96.36.199-24) ... Setting up libfile-ncopy-perl (0.36-1) ... Setting up swish++ (6.1.5-4) ... Processing triggers for man-db (188.8.131.52-2) ... Processing triggers for gnome-menus (3.13.3-9) ... Setting up dlocate (1.07+nmu1) ... Setting up dwww (1.13.3) ... Creating config file /etc/dwww/dwww.conf with new version Building dwww pages in the background... apache2_invoke: Enable configuration dwww
You will notice a few things during the install. First is a list of the additional packages to be installed:
The following additional packages will be installed: dctrl-tools dlocate doc-base info2www libfile-ncopy-perl libmime-types-perl libuuid-perl libyaml-tiny-perl swish++ Suggested packages: debtags rarian-compat doc-debian dpkg-www libmojolicious-perl xpdf-utils antiword The following NEW packages will be installed: dctrl-tools dlocate doc-base dwww info2www libfile-ncopy-perl libmime-types-perl libuuid-perl libyaml-tiny-perl swish++
Then you see the system downloading the packages.
Get:1 http://raspbian-us.ngc292.space/raspbian stretch/main armhf dctrl-tools armhf 2.24-2 [94.4 kB] Get:2 http://raspbian-us.ngc292.space/raspbian stretch/main armhf dlocate all 1.07+nmu1 [23.9 kB] Get:3 http://raspbian-us.ngc292.space/raspbian stretch/main armhf libuuid-perl armhf 0.27-1 [17.6 kB] .......
Next it is unpacking and configuring the packages.
Preconfiguring packages ... Selecting previously unselected package dctrl-tools. (Reading database ... 134811 files and directories currently installed.) Preparing to unpack .../0-dctrl-tools_2.24-2_armhf.deb ... Unpacking dctrl-tools (2.24-2) ...
Finally it is setting things up and configuring the packages.
Processing triggers for mime-support (3.60) ... Processing triggers for desktop-file-utils (0.23-1) ... Setting up dctrl-tools (2.24-2) ... Setting up libyaml-tiny-perl (1.70-1) ... .... Registering 31 doc-base files... Registering documents with dwww... .... Processing triggers for man-db (184.108.40.206-2) ... Processing triggers for gnome-menus (3.13.3-9) ... Setting up dlocate (1.07+nmu1) ... Setting up dwww (1.13.3) ... Creating config file /etc/dwww/dwww.conf with new version Building dwww pages in the background... apache2_invoke: Enable configuration dwww
This is all part of the package management that goes on behind the scenes.
Sometimes you want to use an application that is only available on www.github.com. So lets pretend we want to install the application Adafruit Python GPIO Library
So the first step is to make sure our package repository is up to date.
sudo apt-get update
Next we are going to install some additional tools we will need. Some of these might already be installed.
sudo apt-get install build-essential python-pip python-dev python-smbus git
Next we are going to clone the repository from the github site to our local machine.
git clone https://github.com/adafruit/Adafruit_Python_GPIO.git
Next we will move into the directory containing the install program and run it.
cd Adafruit_Python_GPIO sudo python setup.py install
What we did was install python interface to the GPIO pins on our Raspberry Pi.
When you want to do something on the Raspberry Pi you are going to be writing code. Now you might only need to configure a program that you downloaded. Or you might be creating something from scratch.
The good news is that the number of languages supported on the Raspberry Pi is quite extensive.
Here is a list of languages
Here are a couple of web sites to help you decide what is best for you.
The most common language is Python, for which there plenty of online tutorials.
If your distribution does not contain the language you want you can install it from a repository.
One stumbling block for many users is the idea of users, groups, and permissions. So lets see if we can take some of the mystery out of the equation.
The concept of users is bigger than just who can login to the machine. Lets have a look at the file /etc/passwd
pi@raspberrypi:~ $ cat /etc/passwd root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin bin:x:2:2:bin:/bin:/usr/sbin/nologin sys:x:3:3:sys:/dev:/usr/sbin/nologin sync:x:4:65534:sync:/bin:/bin/sync games:x:5:60:games:/usr/games:/usr/sbin/nologin man:x:6:12:man:/var/cache/man:/usr/sbin/nologin lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin mail:x:8:8:mail:/var/mail:/usr/sbin/nologin news:x:9:9:news:/var/spool/news:/usr/sbin/nologin uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin proxy:x:13:13:proxy:/bin:/usr/sbin/nologin www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin backup:x:34:34:backup:/var/backups:/usr/sbin/nologin list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin systemd-timesync:x:100:103:systemd Time Synchronization,,,:/run/systemd:/bin/false systemd-network:x:101:104:systemd Network Management,,,:/run/systemd/netif:/bin/false systemd-resolve:x:102:105:systemd Resolver,,,:/run/systemd/resolve:/bin/false systemd-bus-proxy:x:103:106:systemd Bus Proxy,,,:/run/systemd:/bin/false _apt:x:104:65534::/nonexistent:/bin/false pi:x:1000:1000:,,,:/home/pi:/bin/bash messagebus:x:105:109::/var/run/dbus:/bin/false statd:x:106:65534::/var/lib/nfs:/bin/false sshd:x:107:65534::/run/sshd:/usr/sbin/nologin avahi:x:108:112:Avahi mDNS daemon,,,:/var/run/avahi-daemon:/bin/false lightdm:x:109:113:Light Display Manager:/var/lib/lightdm:/bin/false epmd:x:110:114::/var/run/epmd:/bin/false
Lets look at the entry for user pi.
The first field is the name of the user pi. The next field, after the colon, is an x. This is a placeholder for the password which is in a separate file, /etc/shadow. Next is the user ID, 1000, followed by the group id, 1000. The three commas are for information like a users name. The next field is the home directory of the user /home/pi. And finally is the program called when the user logs in.
Now many of these users are applications. By having an application run as it’s own user is a security measure, giving each app very limited resources to cause trouble.
You will also notice that most of these apps are attached to the program /bin/false or /usr/sbin/nologin. These are defined as “false - do nothing, unsuccessfully” and “nologin - politely refuse a login”. The false is used for applications that if called by the login app, will exit with a failure exit code. The nologin is used when that user is denied login privileges.
The groups are defined in the file /etc/groups:
pi@raspberrypi:~ $ cat /etc/group root:x:0: daemon:x:1: bin:x:2: sys:x:3: adm:x:4:pi tty:x:5: disk:x:6: lp:x:7: mail:x:8: news:x:9: uucp:x:10: man:x:12: proxy:x:13: kmem:x:15: dialout:x:20:pi fax:x:21: voice:x:22: cdrom:x:24:pi floppy:x:25: tape:x:26: sudo:x:27:pi audio:x:29:pi dip:x:30: www-data:x:33: backup:x:34: operator:x:37: list:x:38: irc:x:39: src:x:40: gnats:x:41: shadow:x:42: utmp:x:43: video:x:44:pi sasl:x:45: plugdev:x:46:pi staff:x:50: games:x:60:pi users:x:100:pi nogroup:x:65534: input:x:101:pi systemd-journal:x:102: systemd-timesync:x:103: systemd-network:x:104: systemd-resolve:x:105: systemd-bus-proxy:x:106: crontab:x:107: netdev:x:108:pi pi:x:1000: messagebus:x:109: ssh:x:110: bluetooth:x:111: avahi:x:112: spi:x:999:pi i2c:x:998:pi gpio:x:997:pi lightdm:x:113: epmd:x:114: ssl-cert:x:115:
The group is used to assign different users to groups to allow access to specific directories and application.
Lets look at the group for netdev:
The first field is the name of the group, netdev, (for network devices). the second field is the password, /etc/gshadow, which is mostly blank. The third field is the group ID, 108. and the last field is a list of the users who belong to the group, pi.
If you are logged in as the user you can see a list of the groups you belong to using this command:
pi@raspberrypi:~ $ id -a uid=1000(pi) gid=1000(pi) groups=1000(pi),4(adm),20(dialout),24(cdrom),27(sudo),29(audio),44(video),46(plugdev),60(games),100(users),101(input),108(netdev),997(gpio),998(i2c),999(spi)
Now lets look at how the users, and groups are related to the files and directories.
Here is a directory long listing from a Raspberry Pi
pi@raspberrypi:~ $ ls -lia /var/lib/nfs total 16 160518 drwxr-xr-x 4 statd nogroup 4096 Nov 13 08:13 . 160382 drwxr-xr-x 50 root root 4096 Jan 21 17:00 .. 160562 drwxr-xr-x 2 statd nogroup 4096 Mar 20 2017 sm 160561 drwxr-xr-x 2 statd nogroup 4096 Mar 20 2017 sm.bak 165084 -rw-r--r-- 1 statd root 0 Mar 20 2017 state
Now let me give you an explaination I found.
-rwxrw-r-- 10 root root 2048 Jan 13 07:11 afile.exe ?UUUGGGOOOS 00 UUUUUU GGGGGG #### ^-- date stamp and file name are obvious ;-) ^ ^ ^ ^ ^ ^ ^ ^ ^ | | | | | | | | \--- File Size | | | | | | | \-------- Group Name (for example, Users, Administrators, etc) | | | | | | \--------------- Owner Acct | | | | | \---------------------- Link count (what constitutes a "link" here varies) | | | | \--------------------------- Alternative Access (blank means none defined, anything else varies) | \--\--\----------------------------- Read, Write and Special access modes for [U]ser, [G]roup, and [O]thers (everyone else) \------------------------------------- File type flag
In my listing the line begins with the file/directory inode number.
The permissions are r = read, w = write, and x = execute. The file type flag of d means a directory. The entries for . is the current directory, and for .. is the parent directory.
So each file and directory can be assigned permissions of rwx for the owner, the group, or everyone.
One important tool on any Linux box is a good text editor. Many of the configurations can be done with a text editor, so you need to have one that you can use easily.
On the Raspberry Pi there are a number of graphical and terminal editors available.
This is a easy to use graphical editor. I find it works well for simple edits.
IDLE is a Python REPL and IDE, so you can write and edit Python code in a window and run it from there.
IDLE has independent windows and syntax highlighting. It’s somewhat buggy, but it’s generally fine for basic use.
Nano is a simple command line editor. It displays help at the bottom of the screen to remind you of it’s basic features.
Vi is one of the oldest editors on any Linux/Unix system. It works only in a terminal. Some people find it difficult since it is a modal editor. It is either in command mode, or edit mode. To insert text you must switch from command to edit mode. To return to command mode you press the ESC key.
I am not showing vi because once you install vim it replaces the vi that comes with the default install.
Interestingly there are 2 editors which can be used in either graphical or commandline versions.
This is an improved version of the vi editor for terminal use. It allows things like syntax highlighting and undo that are not available in vi.
This app does not come installed on the Raspberry PI but you can add it with the command sudo apt install vim. Once you install vim on your Raspberry Pi it replaces the original VI.
This is a graphical version of the Improved VI editor, known a gvim. It gives you the advantages of vim in a graphical version.
This is THE heavy duty programmer’s editor. It has been around for a long time and has a large number of plug in modules. The editor uses a configuration file ~/.emacs to define how it works. In addition it has modules in /usr/share/emacs/site-lisp that add to the editor.
One interesting feature is that is will work in a terminal if called with the -nw option, like emacs -nw test.txt. Or if called without this option it will open a graphical window.
If you are interested in learning more about emacs have a look at the talk Emacs the Swiss Army knife of Linux.
Written by John F. Moore
Last Revised: Mon Jan 28 10:27:50 EST 2019