Skip to content

SystemD

SystemD is the de facto Linux init system since replacing Sysvinit and Upstart in all major distributions. SystemD organizes resources into units, which can be managed by daemons and manipulated by SystemD utilities.

It was designed by a pair of Red Hat developers in 2010 to be a general purpose system manager. It offers parallel execution, explicit dependencies between services, an escape from slow shell scripts, and per-daemon resource control and watchdogs.

Tasks

Scheduling services

Services can be scheduled to start with timers.

sshd.timer
[Unit]
Description=Starts sshd service at beginning of workday, and shuts it down at the end.

[Timer]
Unit=sshd.service
OnCalendar=Mon..Fri *-*-* 09:00:00

[Install]
WantedBy=timers.target

Now, when stopping sshd manually the following output is printed. 

Warning: Stopping sshd.service, but it can still be activated by:
  sshd.timer

The service can be scheduled to shutdown within the service file itself using the RuntimeMaxSec directive.

sshd.service
RuntimeMaxSec=36000 # i.e. 10 hours

This unfortunately will result in the service being reported as failed. This failure can be cleared with this command: 

systemctl reset-failed

Masking

On TrueNAS, the libvirtd socket is masked by default. This means that virsh is not able to connect to the hypervisor until it is unmasked and the service restarted. 
systemctl unmask libvirtd.socket
systemctl restart libvirtd.service
virsh connect qemu:///system

Glossary

Service files

Service files are a type of unit file which have replaced earlier init scripts and describe how to manage a service or application on the server. Active services are placed in /etc/systemd/system, whereas inactive service files distributed with installed packages are placed in /usr/lib/systemd/system.

Docker container as a service: 

[Unit]
Description=Notes Container (Docker)

[Service]
ExecStart=/usr/bin/docker start notes

[Install]
WantedBy=multi-user.target

Slice

A slice unit is a unit configuration file ending in ".slice" which manages resources of a group of processes. SystemD slices implement and build on Linux cgroups.

Slices exist in a hierarchy below the root slice (-.slice) and are used to group scopes and services

  • Scopes contain unrelated processes but not necessarily hierarchically
  • Services are from unit files or Transient Runtime Services and contain processes

Root slices themselves only contain scopes and other slices.

  • user.slice contains all user-related slices and scopes, named after the pattern user-UID.slice

    • session.slice is created for every login session

  • system.slice contain slices, scopes, and services

  • machine.slice contains all container-related slices, scopes, and services.

Services can be assigned to specific slices explicitly by editing the value of the Slice key in the service file. Keys like CPUWeight can assign cgroup resource controls. Other such controls can be viewed in the systemd.resource-control(5) man page. 

[Unit]
Slice=user.slice
CPUWeight=50

SysVinit

SysVInit is the oldest init system used in Linux.

In SysVinit, which used bash scripts to run and manage servicesj, processes were started serially and synchronously, wasting time and system resources. For years, a common mitigation was to run services in the background, simulating concurrency.

Target files

Target files are equivalent to SysVInit runlevels.

SystemD target SysVInit runlevel
poweroff.target 0
rescue.target 1
multi-user.target 3
graphical.target 5
reboot.target 6
emergency.target emergency

Timers

Timer files are systemd unit files with names ending in .timer that control service files. For each timer file, a matching unit file must exist describing the unit to activate when the timer elapses. By default, systemd will search for a service file with a filename matching that of the timer, but failing that a specific unit can be specified with the Unit key within the timer file itself.

Display timers
systemctl list-timers
systemctl status *timer

Like other unit files, timer files may include Unit and Install sections, but must include the Timer section.

Specifying time is done using timestamps which can be monotonic or realtime.

  • Monotonic timers are defined relative to various system hooks using the following directives: OnActiveSec, OnBootSec, OnStartupSec, OnUnitActiveSec, and OnUnitInactiveSec.
  • Realtime timers define timers according to calendar event expressions, denoting real-world dates and times as humans understand them.

Validate timestamps: 

systemd-analyze calendar '*-*-* 00:00:00' --iterations

systemd-run can be used for one-off events as a substitute for anacron. 

systemd-run --on-active=-30sec /bin/touch /home/user/file
This command creates a transient unit file, whose name is provided in the output. 
systemctl cat run-u97.service

Unit files

Unit files are case-sensitive .ini files organized into sections. Unit files can be found in several directories:

  • /lib/systemd/system where the system's copy of unit files are placed by default
  • /etc/systemd/system where unit files override the system default
  • /run/systemd/system where run-time unit definitions are found and given a higher priority than the system default in /lib but lower than that in /etc. These unit files are created dynamically and lost on reboot.

Unit files come in many different types which can be identified by their filename extension (i.e. .service, target, etc.).

Upstart

Upstart was an init system developed by Canonical for Ubuntu meant to replace SysVinit, but it was abandoned in 2014.

Commands

hostnamectl

Permanently change hostname 
hostnamectl set-hostname $HOSTNAME

journalctl

Clean up old logs

journalctl --disk-usage # (3)
journalctl --rotate # (1)
journalctl --vacuum-time=1d # (2)
  1. Ask journal daemon to rotate journal files, immediately archiving and renaming currently active journal files.
  2. --vacuum-size, --vacuum-time, and --vacuum-files can be used singly or in combination to enforce limits on archived journal files.
  3. Show current disk usage of all journal files

Display logs

journalctl -r # --reverse (1)
journalctl -f # --follow (2)
  1. Display output in reverse (newest entries first)
  2. Continuously update the display as new log entries are created

By default, SystemD logs to memory. This can be changed by adjusting /etc/systemd/journald.conf. This requires the directory /var/log/journal to exist.

Persistent logging
[Journal]
Storage=persistent

localectl

Change locale to French 
localectl set-locale LANG=fr_FR.utf8

loginctl

Enable user lingering, which allows users that are not logged in to run long-running services. 
loginctl enable-linger
loginctl show-user | grep Linger - # Confirm

systemctl

Services
systemctl list-unit-files --type=service    # Display all services
systemctl enable --now $SERVICE             # Configure service to start on boot and start it immediately
systemctl status $SERVICE
systemctl is-active $SERVICE 
systemctl disable $SERVICE
systemctl mask $SERVICE                     # Prevent service from being started inadvertently by another process
systemctl restart $SERVICE
Boot targets
systemctl get-default
systemctl set-default graphical.target
systemctl isolate emergency.target          # Change target
systemctl suspend                           # Suspend system

--user specifies the service manager of the calling user. 

systemctl --user enable --now container-notes.service # (1)
systemctl --user status container-notes.service

  1. Here, container-notes.service has been created at ~/.config/systemd/user

systemd-analyze

Check security of a service
systemd-analyze security sshd.service

systemd-cgls

systemd-cgls recursively shows the contents of the selected cgroup hierarchy in a tree.

systemd-delta

Show files that are overridden with systemd.

Display differences among files when they are overridden 

systemd-delta --diff