Small heaps of code

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.


====== System Configuration ======

===== Bootloader =====

System startup works as follows:
  - The BIOS decides on a boot device and loads the boot loader in its Master Boot Record (MBR), typically 512 bytes small.
  - The boot loader (e.g. GRUB2) then starts the Linux kernel (and provides kernel parameters and boot options).
  - Optionally: an init RAM disk (initrd) contains the most important kernel modules to load e.g. RAID controllers or file systems.
  - After the kernel is initialized it starts the mother of all processes: ''init''

==== LILO ====

LILO stores the disk sector where the kernel can be found in the boot loader. Therefore
  * the ''/boot'' partition must be located before sector 1024 of the hard disk, otherwise LILO can't address it;
  * LILO must be reinstalled after each kernel update;
  * it does not matter which filesystem is used, LILO completely ignores it

The main configuration file is ''[[http://linux.die.net/man/5/lilo.conf|/etc/lilo.conf]]''

==== GRUB Legacy ====

GRUB uses two stages:
  - stage1: the 512B in the MBR contain tools to initiate stage2
  - stage2: with tools in ''/boot'' (supported filesystems are e.g. ext2/3/4, reiserfs) the Linux kernel is started

<code bash>
cat /boot/grub/menu.lst         # configuration file (with meta commands that control the behaviour of update-grub)
cat /boot/grub/device.map       # device mapping file
cat /etc/grub.conf              # options, parameters,..
update-grub                     # tool that parses and write menu.lst
</code>

http://wiki.ubuntuusers.de/menu.lst

==== GRUB2 ====

Configuration:
<code bash>
cat /boot/grub/grub.cfg         # generated configuration file (overwritten with update-grub, kernel-updates,..)
cat /etc/default/grub           # main configuration file
ls /etc/grub.d                  # directory with additional configuration files
cat /etc/grub.d/40_custom       # custom entries that update-grub will put in grub.cfg
</code>

With the following commands GRUB2 is installed in two steps - (1) generate a grub.cfg, (2) install GRUB to the MBR. Step 1 must be invoked after each kernel update - otherwise the kernel will not show up as an option during boot.
<code bash>
update-grub                     # generate grub.cfg (searches for installed OSes)
grub-mkconfig -o /boot/grub.cfg # same as update-grub
grub-install                    # install boot loader to MBR or partition
</code>

http://www.gnu.org/software/grub/manual/grub.html
https://help.ubuntu.com/community/Grub2



===== Shared Libraries =====

Shared libraries are used by most programs to save disk space, RAM space and for faster startup times (when library was already loaded by another program). Only programs for minimal environments are statically linked to avoid dependencies.

<code bash>
ldd program             # print required dynamic libraries of an executable
cat /etc/sd.so.conf     # search paths for ldconfig (/lib and /var/lib are implicit)
cat /etc/sd.so.cache    # cache with available dynamic libraries
ldconfig                # update sd.so.cache with libraries found in paths from sd.so.conf and $LD_LIBRARY_PATH
</code>

===== Hostname =====

The host name is persisted in ''/etc/hostname'' and can be shown with ''hostname'' or (using the systemd-way) with ''hostnamectl''.

To change the host name
  - use ''sudo hostname <NAME>'' to set the host name now (would not survive a reboot)
  - edit ''/etc/hostname'' to set it permanently
  - edit ''/etc/hosts'' to properly map the new host name to localhost

Instead of the first two steps you can also use
<code bash>
hostnamectl set-hostname <NAME>
</code>
===== Date & Timezone =====

==== System Time ====

Linux has two clocks: hardware clock (aka BIOS clock) and a software clock (maintained by the Linux kernel). The software clock is initialized with the time of the hardware clock during boot time and written back to the hardware clock on shutdown. [[http://www.tldp.org/LDP/sag/html/keeping-time.html|More details]]

== hwclock ==

The hardware clock can be read and set (by root only) with the ''hwclock'' command. The parameters ''%%--utc%%'' and ''%%--localtime%%'' tell if the hardware clock is / should be set to UTC or local time. 

**Note, however, that hwclock always prints and expects (when setting) date in local time.**

<code bash>
hwclock (--show)         # show current setting
hwclock --systohc --utc  # set hardware clock from system clock (and use UTC)
hwclock --hctosys        # set system clock from hardware clock
</code>

Completely manual usage:
<code bash>
hwclock --set --date="2012-04-19 16:45:05" --utc
hwclock --set --date="2012-04-19 16:45:05" --localtime
</code>

The current time zone of the hardware clock is specified in ''/etc/adjtime'', or UTC if the file is missing.

== date ==

Changing the system clock (time + date)
<code bash>
date -d "2012-12-12T12:00" # print a date
date -s "2012-12-12T12:00" # set a date + time as system time
date +%T -s "14:30"        # only set the system time (not the date)
</code>

The string format interpreted as date is extremely versatile and described in ''info date'', here are some examples:
<code bash>
date -d @1234567890       # Unix time / seconds since epoch
date -d "2014-01-01"      # ISO 8601
date -d "next Thursday"
date -d "2 years"         # now + 2 years
date -d "2 years ago"
</code>

The output format can be adjusted with format strings:
<code bash>
date +'%s'                # Unix time / seconds since epoch
date +'%Y-%m-%dT%H:%M:%S' # in ISO 8601
</code>

=== Network Time Protocol (NTP) ===

The best idea is to set the hardware clock via BIOS and let an NTP daemon take care of adjusting the system time. [[https://help.ubuntu.com/14.04/serverguide/NTP.html|Ubuntu preinstalls]] ''ntpdate'', but the more advanced ''ntpd'' is also available.

NTP synchronizes your system time with the ones from time servers (if it's not off by more than a few minutes). The NTP pool at ''pool.ntp.org'' provides a fail-safe address because it actually uses many different servers behind this address.

== ntpdate(-debian) ==

''ntpdate'' is the standard tool to synchronize the software clock with NTP once. It requires a server to be provided at each call.

''ntpdate-debian'' is a slightly modified version that can be configured via ''/etc/default/ntpdate''. Ubuntu executes this program every time a network becomes available, see ''/etc/network/if-up.d/ntpdate''.

<code bash>
sudo ntpdate-debian                # get NTP time from the configured server
sudo ntpdate pool.ntp.org          # get NTP time from a reliable cluster of NTP servers
</code>

== ntpd ==

The ntp daemon always runs in the background and can correct drifts slowly to avoid jumps in time.

It can be configured (e.g. change the NTP servers) via ''/etc/ntp.conf''.

''ntpq'' can query the status of the ntp daemon:
<code bash>
ntpq -p                            # print ntp peers & their state
</code>
==== System Time Zone ====

The system time zone is set in two places (which should match):
  * ''/etc/localtime'' contains the binary time zone specification (one of the files in ''/usr/share/zoneinfo'')
  * ''/etc/timezone'' contains the human-readable POSIX timezone name

== tzselect ==

''tzselect'' is an interactive program to let a user retrieve the appropriate POSIX timezone string for his location, e.g. "Europe/Vienna" for Austria. It does not change the time zone!

=== Setting via tzconfig ===

''tzconfig'' was used to set the timezone, but on modern Debian / Ubuntu systems we use ''dpkg-reconfigure'', which will run through the same questions as ''tzselect'':
<code bash>
dpkg-reconfigure tzdata
</code>

This program copies the appropriate (binary) time zone info file from ''/usr/share/zoneinfo'' to ''/etc/localtime''.

=== Other methods ===

Set the environment variable ''TZ'' to the appropriate timezone name, e.g. in ~/.profile
<code bash>
TZ='Europe/Vienna'; export TZ
</code>

[[http://www.wikihow.com/Change-the-Timezone-in-Linux|Still more methods...]]

==== Quick Fix ====

Quickly fix the system time & timezone:
<code bash>
dpkg-reconfigure tzdata      # set proper timezone
ntpdate-debian               # get current time via NTP
hwclock --systohc --utc      # set hardware clock to (UTC) time retrieved from via NTP 
</code>

===== Locales =====

[[http://www.gnu.org/software/gettext/manual/html_node/Locale-Environment-Variables.html#Locale-Environment-Variables|Locales]] enable internationalization and define e.g. number formats (LC_NUMERIC) or date and time formats (LC_TIME). 

The default is set with LANG, which can be overriden with the more specialized LC_*. LC_ALL overrides everything else, and should not be set except for the execution of commands.

<code bash>
locale                  # print current locale settings
locale -a               # print names of available locales
locale -k LC_TIME       # print current formats for a locale setting
locale charmap          # print currently used charmap (e.g. UTF-8)
</code>

==== Generate locales ====

All available locales are listed in ''/usr/share/i18n/SUPPORTED''. To use a locale it must be generated first with ''locale-gen'':

<code bash>
locale-gen de_AT.UTF-8
</code>

Note, that locale-gen fails silently if an invalid locale is given. (return code 1)


==== Permanent settings ====

Locale variables are set in the configuration file ''/etc/default/locale''. A great configuration for using the English language with ISO date/time format and european paper size (A4) is:
<code>
LANG="en_US.UTF-8"
LC_TIME="en_DK.UTF-8"
LC_MEASUREMENT="en_DK.UTF-8"
LC_PAPER="de_AT.UTF-8"
</code>

**Note**, that some programs ignore the settings in LC_PAPER and use ''/etc/papersize'' instead. Set a consistent format there as well, e.g. ''a4''.

**Note for KDE:** the locale configuration tool will create the file ''~/.kde/env/setlocale.sh'', which overrides ''/etc/default/locale'' - delete undesired settings there.

==== Temporary settings ====

To set a certain locale for one command only:
<code bash>
LC_TIME="en_DK.UTF-8" locale -k LC_TIME        # use specific locale for one command
</code>

It may also be useful to override all locale settings and use the most basic locale available on every machine: ''C'' or ''C.UTF-8'': 
<code bash>
LC_ALL=C.UTF-8 locale -k LC_TIME
LC_ALL=C.UTF-8 ssh host
</code>

[[http://unix.stackexchange.com/questions/87745/what-does-lc-all-c-do/87763#87763|More in-depth explanations]]

====== Partitioning & File Systems ======

In order to use a hard disk to install an operating system or store files it has to be
  - [[http://www.tldp.org/HOWTO/Partition/intro.html#explanation|partitioned]] and
  - each partition must have a [[http://tldp.org/LDP/intro-linux/html/sect_03_01.html|file system]]

A special case are [[http://www.tldp.org/HOWTO/Partition/setting_up_swap.html|swap partitions]] (or files), which is basically very very slow RAM (memory) on the hard disk.

===== Filesystem Hierarchy Standard (FHS) =====

The [[http://www.pathname.com/fhs/pub/fhs-2.3.html|Filesystem Hierarchy Standard (FHS)]] defines a common directory structure for Unix based OSes, which is important to be kept in mind during partitioning.

FIXME outline the basics:
  * /bin, /etc, /dev, /lib, /proc, /sbin: must be contained in one partition (used as root)
  * /boot, /home, /opt, /root, /tmp, /usr, /usr/local, /var: can be on different partitions

===== Creating Partitions & Filesystems =====

== Partitioning ==
The programs ''fdisk'' and ''cfdisk'' can create a / alter the partition table in PCDOS format for disks up to 2TB.

Common partition IDs are: 5=extended, 7=ntfs, 82=swap, 83=linux.

<code bash>
fdisk /dev/sda                  # start partitioning of /dev/sda
fdisk -l /dev/sda               # list all partitions of /dev/sda
cfdisk /dev/sda                 # use a curses GUI for paritioning
</code>

For disks bigger than 2TB typically GPT - the [[http://en.wikipedia.org/wiki/GUID_Partition_Table|GUID_Partition_Table]] - is used, which can be created with GNU ''parted''. Read [[http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/|this good tutorial about disks with 4096 byte sectors]]

== Create a file system ==

<code bash>
mke2fs                          # create an ext2/3/4 file system
mkfs                            # create a file system (ext and non-ext file systems) 
mkfs.ext4                       # mkfs.* are frontends that call the relevant programs to create a specific file system
</code>

The following two commands can be used to create an ext4 file system. The parameter '-n' can be used to simulate the action without actually changing anyting.
<code bash>
mke2fs -t ext4 /dev/sda1
mkfs.ext4 /dev/sda1
</code>

== tune2fs - adjusting filesystem parameters for ext2/3/4 ==
<code bash>
tune2fs -l <device>             # show stats & current value of options
tune2fs -L <name> <device>      # set volume name
tune2fs -i <interval> <device>  # interval between checks n[d|w|m], by default in days, e.g. 6m
tune2fs -c <count> <device>     # maximum mount-count between checks
</code>


===== Mounting =====

The ''findmnt'' command and various files can be used to inquire the currently mounted devices and swap spaces.
<code bash>
lsblk                  # list block devices (i.e. no network or virtual devices)
findmnt                # show all currently mounted devices
findmnt -t ext4,cifs   # show mounted devices with a certain file system type
mount                  # show all currently mounted devices (deprecated!)
cat /etc/mtab          # ..
cat /proc/mounts       # ..
cat /proc/swaps        # show currently used swap spaces
</code>

The main configuration file is ''/etc/fstab'':
<code bash>
cat /etc/fstab         # configuration of filesystems to mount (specifies location, file system type,..)
mount -a               # mount all devices in /etc/fstab except the ones set to 'noauto'
umount -a              # unmount all devices in /etc/fstab except the ones set to 'noauto'
</code>

The previously mentioned invocations and config files cover most of the daily use cases. More advanced use cases follow.

== Manually mount a CDROM ==

<code bash>
mount -t iso9660 /dev/cdrom /mnt/cdrom
</code>

== Manually mount an ISO file ==

<code bash>
mount -o loop -t iso9660 foo.iso /mnt/mountpoint
</code>

== Manually mount a partition ==

Mount an ext4 partition with some restrictive mount options. See FILESYSTEM INDEPENDENT MOUNT OPTIONS in ''man mount'' for an explanation of all mount options.
<code bash>
mount -t ext4 -o nodev,noexec,noatime,nosuid /dev/sda1 /mnt/mountpoint 
</code>

== Remount a partition read-only ==

This is e.g. useful for using ''fsck'' on a partition. Note, that this overwrites all mount options if the partition has been mounted manually - if there is an entry in ''/etc/fstab'', ro will be added to the mount options specified there.

<code bash>
mount -o remount,ro /dev/sda1 /mnt/mountpoint
</code>

== Mount certain file systems only ==
When combining ''-a'' with  ''-t'' all file systems of a certain type in ''/etc/fstab'' can be selected (except if they are noauto of course). In this example we force (-f) unmount all cifs filesystems (Windows / Samba shares) and then mount them again.

<code bash>
umount -a -f -t cifs
mount -a -t cifs
</code>


===== Check & Monitor File Systems =====

The family of ''[[http://www.thegeekstuff.com/2012/08/fsck-command-examples|fsck.*]]'' (e.g. ''fsck.ext4'') commands is used to check & monitor filesystems.

**Note:** file systems must be unmounted or at least mounted read-only, because ''fsck'' does not only check but also repairs the file system! Repairing a mounted file system can lead to severe problems.

<code bash>
fsck -N <device>                # dry run (only shows you what would be done) on a device, e.g. /dev/sda1
fsck -ARM                       # check all file systems in /etc/fstab, except root, and except all mounted ones
fsck -t ext4 -p <device>        # check a device with an ext4 filesystem and repair errors automatically (p=preen); same as fsck.ext4 -p
</code>

These two tools can be used to further inspect your ext file systems, but be careful with ''debufs''!

<code bash>
debugfs <device>                # ext2/ext3/ext4 interactive file system debugger (use with care!)
dumpe2fs <device>               # dump ext2/ext3/ext4 filesystem information (super block & group information of blocks)
</code>

===== Disk Health =====

==== badblocks ====

Badblocks is a utility to check disk health with either the non-destructive read-mode or the destructive (all data will be erased) write-mode.

<code bash>
sudo badblocks -wvs /dev/sdb
</code>

==== S.M.A.R.T. Monitoring ====

With ''smartctl'' (contained in the package ''smartmontools'') you can check the health status of a hard disk.

<code bash>
sudo smartctl /dev/sda -H     # quick summary to see if HDD is OK or not
sudo smartctl /dev/sda -a     # print all SMART information
</code>

When printing all information the attributes are shown. VALUE and WORST should be **above THRESH** for the disk to be OK.

<code>
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       1305
  3 Spin_Up_Time            0x0027   141   141   021    Pre-fail  Always       -       3916
  ...
</code>
===== Swap =====

<code bash>
swapon -s      # see currently active swap partitions
</code>

<code bash>
mkswap         # set up swap (on a device or in a file, which must be created with dd)
</code>

== (De)activate swap ==
For 'mounting' swap partitions or files extra commands are used:

<code bash>
swapon -a      # activate all swap partitions in /etc/fstab (except those with noauto set)
swapoff -a     # deactivate ..
</code>

Hint: use ''swapoff'' to move all memory back into RAM and to get a responsive system quickly (after the process that caused swapping is closed).


== How to use a swap file ==
Create a 64M swap file in the current directory & check that it's activated
<code bash>
dd if=/dev/zero of=swapfile bs=1024 count=65536
chmod 600 swapfile
mkswap swapfile
swapon swapfile
cat /etc/swaps
</code>


====== System Startup ======

===== Runlevels  =====

Runlevels were used in older distros to only partially start up the system. From single-user mode without networking, to multi-user mode with networking. Nowadays most distros don't use many different runlevels much anymore.

Runlevels: 
  * 0: halt
  * S: single user mode (often: 1)
  * 2-5: differs between distros, Ubuntu uses 2 as default
  * 6: reboot

== runlevel ==
The previous and current runlevel can be checked with ''runlevel''. N stands for no previous runlevel.

== telinit ==
Switching runlevels is done with ''telinit''. In most distros ''init'' does the same.

<code bash>
telinit 2        # switch to runlevel 2
telinit 6        # reboot system
</code>

== shutdown ==

With ''shutdown'' reboots or shutdowns can be timed with either 'now', +<minutes> (e.g. +10), or hh:mm (e.g. 19:20).
Other logged in users will be notified of a (pending) shutdown, no logins are possible five minutes before shutdown, and it is possible to specify a customized (optional) message.

<code bash>
shutdown -h now                                  # shutdown (halt or power off) now
shutdown -P now                                  # shutdown and power off now
shutdown -r +10 "message"                        # reboot in 10 minutes and send message
shutdown -k 20:00 "possible power outage at 8"   # 'fake' a shutdown. messages are sent, but no shutdown will be done
shutdown -c                                      # cancel pending shutdown
</code>

== reboot, halt, poweroff ==

An alternative to the shutdown command are the following commands:

<code bash>
reboot
halt           # shut down OS, but do not power off (via ACPI)
poweroff       # shut down OS and power off (what you usually want)
</code>
===== Init Daemons =====

The binary ''/sbin/init'' is started first (hardcoded into the kernel) and therefore the mother of all processes.
It starts all required processes during boot.

The 'original' Sysvinit is replaced in most distros as of 2013 with init systems that are event based (asynchronous).

== sysvinit ==

Configuration via
<code bash>
/etc/init.d       # directory with startup scripts (nowadays still: legacy scripts)
/etc/inittab      # set default runlevel
</code>

== upstart ==

Upstart is a Ubuntu-only event-based replacement for the original init daemon and brings some new tools and commands, but was replaced with systemd since Ubuntu 15.05.

Init scripts and scripts for services are contained in ''/etc/init''. Services can be controlled like this:

<code bash>
service myservice start       # equivalent: start myservice
service myservice stop        # equivalent: stop myservice
service myservice reload      # equivalent: reload myservice
service myservice restart     # equivalent: restart myservice
service --status-all          # get status of all services
</code>

== systemd ==

Modern event-based replacement for ''init'', used by most distributions in 2015.

Init scripts are represented as [[https://www.freedesktop.org/software/systemd/man/systemd.service.html|service units]] in the form of ''.service'' files, which can be found in ''/lib/systemd/system'' (preinstalled) and ''/etc/systemd/system'' (to be edited by the admin).

Example of a ''.service'' file with the the three sections:
<code>
[Unit]
Description=Foo

[Service]
ExecStart=/usr/sbin/foo-daemon

[Install]
WantedBy=multi-user.target
</code>

Starting and stopping services works as follows:
<code bash>
systemctl start myservice
systemctl stop myservice
systemctl status myservice
</code>

Logs of a service can be accessed via ''journalctl'':
<code bash>
journalctl -u myservice
</code>
====== Package Management ======

The Linux way of (un)installing programs.

===== Debian (.deb) =====
Quick overview of commands & config files involved in Debian:
<code bash>
dpkg                        # simple utility to handle single packages
apt-get                     # install / remove / update packages (uses dpkg)
apt-mark                    # hold / unhold packages (aka apt pinning)
apt-cache                   # cache for searching packages
dselect                     # curses GUI for selecting and installing programs
aptitude                    # curses GUI and command line tool that unites apt-get, apt-cache and dselect
cat /etc/apt/sources.list   # main config file containing addresses for package sources
ls /etc/apt/sources.list.d  # directory with more config files containing addresses for package sources
</code>

When not using ''aptitude'' the following commands are tpically used. Otherwise simply replace ''apt-get'' and ''apt-cache'' with ''aptitude''.
<code bash>
apt-get update              # synchronize list of available packages with server
apt-cache search            # search for a package (partial name is OK)
apt-get upgrade             # upgrade upgradable packages (not if new dependencies are requird)
apt-get dist-upgrade        # upgrade upgradable packages (and install new dependencies if required)
apt-get install             # install packages (exact name required)
apt-get remove              # remove packages (exact name required) but leave config files
apt-get purge               # remove packages (exact name required) and config files
</code>

== Advanced Usage==

Some more advanced usage examples:
<code bash>
apt-cache depends package   # show dependencies (and suggestions) for a package
dpkg -l                     # list names of all installed packages
dpkg -L package             # list all files (full path) contained in a package
dpkg -s package             # show status of package (e.g. is it installed or not)
dpkg -S file                # show packages that contain this file (matching by name)
</code>


== alien ==
''alien'' is a package converter that can convert ''.deb'' packages into ''.rpm'' and vice versa.

===== Red Hat (.rpm) =====

There is only one relevant tool for package managemnt in Red Hat: ''rpm''. It has five basic modes:

<code bash>
rpm -i             # --install
rpm -U             # --upgrade
rpm -e             # --uninstall (erase)
rpm -V             # verify
rpm -q             # --query
</code>

Flags for (un)installing and upgrading:
<code>
--force            # force install/update, same as --install --force
--no-deps          # (un)install/update even if dependencies are not satisfied
--test             # just test and don't make any changes
</code>

Flags for querying:
<code>
-a  or  --all      # list all installed packages
-f  or  --file     # list the package a file belongs to
-l  or  --list     # list all files in a package
-i                 # information about an installed package
-p <package>.rpm   # query regarding the specific package-file
</code>

== Advanced Usage==

Some more advanced usage examples:
<code bash>
rpm -qR package             # show dependencies for a package
rpm -qa                     # list names of all installed packages
rpm -qlp package.rpm        # list all files (full path) contained in a package
rpm -qf file                # show packages that contain this file (matching by name)
</code>

== Extract files from .rpm files ==

rpm packages can not be unzipped directly, but this can be achieved by using ''rpm2cpio'' and ''cpio''. The following command extracts an rpm into the current directory:

<code bash>
rpm2cpio package.rpm | cpio -idmv
</code>

==== YUM ====

The 'yellowdog updater, modified' aka ''yum'' is a package management tool used in many distributions based on .rpm packages. Its usage strongly resembles ''aptitude'':

<code bash>
check-update                # synchronize list of available packages with server
yum search                  # search for a package (partial name is OK)
yum update                  # update all packages
yum install                 # install packages (exact name required)
yum remove                  # remove packages (exact name required) but leave config files
</code>

== yumdownloader ==

''yumdownloader'' downloads RPMs from Yum repositories. The relevant configuration file is ''/etc/yum.conf'' and the files in ''/etc/yum.repos.d''.


====== User Management ======

== user and group ids - uid & gid ==
Root always has uid 0, uids from 1 until e.g. 499 (RHEL) or 999 (Debian) are reserved for system use. Uids above that are used for regular users, e.g. 1000 for the first user on Debian.

The same goes for gid: root has gid 0,.. 

Every user has at least one group, the first group being the primary group.

== /etc/passwd ==

User details are stored in ''/etc/passwd''. The columns (separated by colons) are:
  - login
  - optional encrypted password (or 'x')
  - uid (numerical user ID)
  - gid (numerical group ID)
  - comment (e.g. user name or daemon description)
  - home
  - shell (optional, set to ''/bin/false'' if no login should be possible)

Example:
<code>
markus:x:1000:1000:markus,,,:/home/markus:/bin/bash
</code>

== /etc/shadow ==

Normally the password is not stored in ''/etc/passwd'', but in ''/etc/shadow'', which should only be readable by root.
''/etc/shadow'' consists of these colon-separated fields:
  - login
  - encrypted password (prepend with '!' to disable login) 
  - date of last password change (days since epoch)
  - minimum password age (days)
  - maximum password age (days)
  - password warning period (days)
  - password inactivity period (days) (how many days after a password change is necessary is the old password still accepted)
  - account expiration date (days since epoch)
  - reserved field

All fields except the first two can be left blank, thereby disabling password aging.

Example (with shortened password hash):
<code>
markus:$6$d0uFLcE...AMuZRV0:16560:0:99999:7:::
</code>

Get a human-readable representation of the expiration information with ''chage'':
<code bash>
chage -l user
</code>

== /etc/group ==

''/etc/group'' consists of these colon-separated fields:
  - group name
  - encrypted password
  - gid
  - members (comma-separated list of users)

== /etc/gshadow ==

As for users the group password is normally not stored in ''/etc/group''.
Note, that the members in these two files are stored redundantly and must match.

''/etc/gshadow'' consist of these colon-separated fields:
  - group name
  - encrypted password
  - administrators (comma-separated list of users that can change the group password)
  - members (comma-separated list of users)

== vipw ==

Preferrably edit these important files with these tools to avoid concurrent editing and broken files (e.g. due to forgetting a colon).

<code bash>
vipw         # edit /etc/passwd
vipw -s      # edit /etc/shadow
vigr         # edit /etc/group
vigr -s      # edit /etc/gshadow
</code>

== getent ==

With ''getent'' you can "get entries from Name Service Switch libraries", e.g. the passwd, group, shadow, gshadow databases. (quite similar to just greping the respective files)

<code bash>
getent passwd username
</code>

== /etc/skel ==

This directory serves as a template for home directories. When a new user account is created, the home directory will be populated with the contents of ''/etc/skel''.


===== Add, remove, change users =====

The three standard tools are ''useradd'', ''userdel'' and ''usermod''.

==Create a user==
<code bash>
useradd name                # create a new user (without password; home directory is set, but not created)
useradd -m name             # also create a home directory (with contents of /etc/skel)
useradd -s /bin/bash name   # assign a shell
useradd -r name             # create a system user (uid <1000)
useradd -g gid -u uid name  # override default gid and uid (-o even allows uid-duplicates)
</code>

The password can be set with ''passwd'' or the ''-p'' option (but ''-p'' already requires the encrypted form to be stored in /etc/shadow)

==Delete a user==
<code bash>
userdel name                 # delete a user
userdel -f -r name           # force delete (even when logged in) a user and delete its home directory (-r)
</code>

== Change basic user account details ==
''usermod'' allows to change the parameters also available in ''useradd'' (and more):

<code bash>
usermod -u uid -g gid name      # change uid and gid for an existing user (ownerships in home directory are adapted as well)
usermod -d /home/newhome name   # change home directory (created if it does not exist)
usermod -md /home/newhome name  # change home directory and move all files from the old home
usermod -l newname name         # rename / change login of a user (does __not__ change the home directory)
</code>

===== Add, remove, change groups =====

The tools ''groupadd'', ''groupdel'' and ''groupmod'' work similar to the user-counterparts:

==Create a group==
<code bash>
groupadd gname               # create a new group
groupadd -r gname            # create a system group (uid <1000)
groupadd -g gid gname        # override default gid (-o even allows uid-duplicates)
</code>

==Delete a group==
<code bash>
groupdel gname               # delete a group
</code>

==Modify a group==
<code bash>
groupmod -g gid -n newgname gname  # change gid and name of a group
</code>

=== Managing groups of a user ===

View and add groups for a user:
<code bash>
groups user                          # show all groups a user is part of
usermod -a -G group1,group2 user     # add user to comma-separated group(s)
</code>

Remove or exlicitily set groups for a user:
<code bash>
gpasswd -d user gname                # remove user from group
usermod -G "gname1,gname2" user      # explicitly set comma-separated group(s) for a user
vigr                                 # open /etc/groups in vi (locks the file & enforces correct format on save)
</code>

==== Group password ====

Groups can be assigned a password, which is stored in ''/etc/gshadow'' (or ''/etc/group''). Group membership can then be temporarily acquired.

<code bash>
gpasswd gname                  # assign password to a group
newgrp gname                   # temporarily get membership for group
</code>

Group administrators are allowed to change/remove the group password and add or remove users to the group. They can be assigned as follows:
<code bash>
gpasswd -A "user1,user2" gname # assign administrator users to group
</code>

===== (Un)locking Users =====

== Lock a user ==
Locking is done (by convention) by prepending a '!' to the password in either ''/etc/passwd'' or ''/etc/shadow'', or one of these two commands:
<code bash>
usermod -L user
passwd -l user
</code>

Or by deleting the currently set password (this can not be undone)
<code bash>
passwd -d user
</code>

== Unlock a user ==
<code bash>
usermod -U user
passwd -u user
</code>

===== User Account Limits =====

==== Account Expiration ====

The fields in ''/etc/shadow'' regarding passwords age or account expiration can be manipulated with ''chage''.

After a password has been expired the user is forced to chage it at the next login. If an inactivity period is specified accounts with expired passwords will be locked after this period.

<code bash>
chage -m min -M max user        # set minimum and maximum nr of days between password changes
chage -I days user              # set inactivity period in days
chage -d 2014-01-01 user        # manually set date of last passwort change
</code>

An account can also have a fixed expire date:
<code bash>
chage -E 2014-01-01 user        # set expiration date of account, afterwards no login is possible (accuracy = days)
</code>

Passing -1 as value will erase the limits.

==== Limits ====

With the bash built-in ''ulimit'' resources allocated to a user can be limited for the current shell.
<code bash>
ulimit -a            # show all limits for the current user
</code>

With '-S' soft and '-H' (or no extra parameter) hard limits can be set. Soft limits can be raised up to the hard limit by the user. The hard limit can only lowered by the user. 

<code bash>
ulimit -S -u 100     # set a soft limit for the maximum nr of processes
ulimit -n 50         # set soft & hard limit for the maximum nr of open files
</code>

A good place to set ''ulimit''s is ''/etc/profile'', or in the config file ''/etc/security/limits.conf'', where the configuration can be specific to a user or group (prepend @):
<code>
#<domain>      <type>  <item>         <value>
user           hard    nproc          100
@group         hard    nofile         50
</code>
==== sudo & su ====

''sudo'' allows a normal user to execute commands as other user (also super user), with ''su'' the current user id can be changed to become super user (a root shell).

<code bash>
sudo passwd otheruser          # set password for otheruser usint root privileges
sudo -u www-data mkdir dir     # create a directory as user www-data
</code>

Who (or which groups) are allowed to do so is specified in ''/etc/sudoers'', which must be edited with ''visudo'' and roughly follows the format ''user hostlist=(userlist) commands'':
  * user: which user the permissions are for
  * hostlist: on which host (when sharing the sudoers on several hosts)
  * userlist: which identities the user may take
  * commands: which commands may be executed

Simple example:
<code>
# User privilege specification
root    ALL=(ALL:ALL) ALL
# Members of the admin group may gain root privileges
%admin  ALL=(ALL) ALL
# Allow members of group sudo to execute any command
%sudo   ALL=(ALL:ALL) ALL
</code>

The format is quite complex, have a look in ''man 5 sudoers''.
<code>
#On the ALPHA machines, user john may su to anyone except root 
# but he is not allowed to specify any options to the su(1) command.
john    ALPHA = /usr/bin/su [!-]*, !/usr/bin/su *root*
</code>

===== User Monitoring =====

==== Who is/was logged in? ====
The programs ''w'' and ''who'' can show who is currently logged into the system.

<code bash>
w               # print users who are logged in and their processes
who             # print information about users who are logged in
who -b          # time and date of last system boot
who -r          # print current runlevel
</code>

''last'' prints the last log ins (user, time,..).
<code bash>
last            # all recent logins
last username   # last logins of user
last tty1       # last logins on tty1
</code>

==== Who holds open files / ports? ====

''fuser'' and ''lsof'' (list open files) can check which users or processes opened a file, file systems or ports. 
**Note**, that these programs should be **run as root** to get a full report (especially when checking ports)!
<code bash>
lsof                        # list all files opened by active processes

fuser /tmp/file             # show process ids accessing a file
fuser -v /tmp/file          # show verbose info (including user, command,..)
lsof /tmp/file

lsof +D /tmp                # list all processes that have opened a file in /tmp

fuser -m -u /mnt/usb1       # show process ids (and the user name: -u) accessing a mount point / file system
lsof /mnt/usb1

fuser -n tcp 80             # show process ids accessing port 80 (equivalent: fuser 80/tcp)
lsof -i tcp@localhost:80

lsof -p 1000                # show files opened by process with id 1000

lsof -u markus              # show files opened by user markus
</code>

''fuser'' can also kill the identified processes, which can be useful e.g. when trying to unmount a file system.
<code bash>
fuser -m -k /mnt/usb1          # send SIGKILL (default for -k) to all processes accessing /mnt/usb1
fuser -m -k -SIGTERM /mnt/usb1 # send SIGTERM (see fuser -l for supported signals)
</code>

===== Users managing themselves =====

A normal user can not use ''usermod'', but has the following possibilites.

== Change password ==

<code bash>
passwd                   # interactively change password
</code>

== Change login shell ==

<code bash>
chsh                     # start interactively
chsh -s /bin/bash        # change login shell
</code>

== Get account & password information ==

Check e.g. the account or password expiry date with:
<code bash>
passwd -S                # oneliner
chage -l user            # more verbose
</code>


===== Notifying Users =====

==issue==
Before login the contents of ''/etc/issue'' are printed.

==motd==
After login users can be greeted with a message of the day in ''/etc/motd''.

==wall==
The command ''wall'' can send warning messages to all users logged into a machine.
<code bash>
wall
The system has to be shut down in 5 minutes, please save your work and log out.
<Ctrl D>
</code>

==nologin==
In case an administrator wants to temporarily forbid logins of other users the file ''/etc/nologin'' can be created. It allows no other users than root to log in (also via SSH,..). **Note**, that this is dangerous and not recommended if root has no password set (as per default in Ubuntu).
===== Debian's Convenience Tools =====

Debian provides two convenience programs for adding and deleting users and groups.

However, for further tweaking of users and groups the default ''usermod'' tool must be used.

==== Adding users and groups ====

By default ''adduser'' and ''addgroup'' automatically create fully usable accounts (e.g. by creating a new home directory), asks for a password and a detailed description and assures that the gid and uid conform to Debian standards. The defaults can be changed in ''/etc/adduser.conf''.

These simple invocations create normal (non-system) users or groups:
<code bash>
adduser name
addgroup name
</code>

The description field in ''/etc/passwd'' is to sused tore these properties (comma-separated):
  - full name
  - room number
  - work phone
  - home phone
  - other


==Create a system user that can not be used to log in==
<code bash>
adduser --system --disabled-login name
</code>


==== Deleting users and groups ====

These tools remove users and groups by removing the entries from the relevant config files, but leaves the home directory intact (by default - see ''/etc/deluser.conf'').

<code bash>
deluser name
delgroup name
</code>

Home directories can however be removed or archived by providing a simple argument
<code bash>
deluser --remove-home name       # remove home directory and mail spool
deluser --remove-all-files name  # remove all files owned by user (system-wide)
</code>

When either the home or all files are removed a backup archive can be created as well:
<code bash>
deluser --backup --backup-to /tmp --remove-home name
</code>

==== Remote login via Secure Shell (SSH) ====

[[https://www.debian-administration.org/article/530/SSH_with_authentication_key_instead_of_password|Log in]] to a remote host with ''ssh username@host''.

To use authentication keys instead of passwords:
<code>
ssh-keygen                                      # generate your public and private key (~/.ssh/id_rsa)
ssh-copy-id -i ~/.ssh/id_rsa.pub username@host  # copy it to the ~/.ssh/authorized_keys on the remote machine
</code>
Small heaps of code

User Tools

Site Tools

Page Tools
