Small heaps of code

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== System Logs ======

Log files, which are of great of importance for troubleshooting, are typically found in ''/var/log''. Some very common log files and their content (apart from log files or directories created by some applictions like ''/var/log/apache2'' are:

  * ''/var/log/auth.log'' authorization log: use of sudo, ssh,...
  * ''/var/log/daemon.log'' daemon log
  * ''/var/log/debug.log'' syslog messages with priotity DEBUG (& Ubuntu debugging messages)
  * ''/var/log/kern.log'' kernel log
  * ''/var/log/messages.log'' syslog messages with priority INFO
  * ''/var/log/syslog'' most syslog messages

The **kernel ring buffer** is important, because it allows logging even in the startup phase when it is not yet possible to log to regular files in ''/var/log''. It can be read directly with the command ''dmesg''. Ubuntu also writes these messages to a log file if possible:
  * ''/var/log/dmesg'' logs from kernel ring buffer

[[https://help.ubuntu.com/community/LinuxLogFiles|More Ubuntu specific info.]]


===== syslogd =====

The original ''syslogd'' is nowadays (2014) mostly replaced by ''rsyslogd'' (e.g. Debian) and ''syslog-ng'' (e.g. SuSe).

The original config file for logging rules was ''/etc/syslog.conf''. For rsyslogd it is ''/etc/rsyslog.conf'', in Ubuntu configuration is further split into several files: logging rules are moved to ''/etc/rsyslog.d/''.

==== Configuration with Selectors ====

Original syslogd-style selectors (are still supported in rsyslogd) and consist of a **facility** (e.g. auth, cron, syslog, user1,..) and **priority** (e.g. debug, warning, err, alert,..), which together form a **selector**, and an **action**, e.g. logfile.

The selector is written in the form 'facility.priority', where either can be replaced by a star (*) - meaning all. The meaning of a selector is, that all logs from the facility with the given priority or higher will be logged. To select exactly one priority you can add an equals sign as such: 'facility.=priority'.

The action (or target) can be a regular file, named pipes, terminals or consoles, remote machines,.. Prepending the target with a minus (-) will make it buffered.

<code>
# print kernel warnings (and worse) on a konsole
kern.warn               /dev/tty10

# print kernel warnings (and only warnings) on a konsole
kern.=warn               /dev/tty10

# log warnings (or worse) of several facilities
local0,local1.warn      /var/log/localwarnings.log
local0,local1.warn      -/var/log/bufferedlocalwarnings.log

# log everything but auth & authpriv to syslog file
*.*;auth,authpriv.none  -/var/log/syslog

# print emergency messages on the terminals of all users
*.emerg                 :omusrmsg:*
</code>

**Note**, that if syslog should write to terminals or consoles it must have root privileges. Therefore no privilege drops must be activated in the config file, e.g. ''$PrivDropToUser syslog'' (which is default in Ubuntu).

See also [[http://www.rsyslog.com/doc/master/configuration/filters.html|rsyslog documentation on filters]].

===== klogd =====

''klogd'' is similar to ''syslogd'' but only logs messages from the kernel and can do so in more detail. [[http://de.linwiki.org/wiki/Linuxfibel_-_System-Administration_-_Protokollierung#Der_klog-D.C3.A4mon|See this page for more info (German!)]]

===== logger =====

''logger'' can be used to conveniently log with syslogd, rsyslogd,.. e.g. in bash scripts. In addition to syslog it can also log to stderr, files and more.

<code bash>
logger "message"                        # log to syslog (user.noticy by default)
logger -p local0.warn "message"         # explicitly specify facility
logger -s "message"                     # also log to stderr
logger -f /tmp/extralog "message"       # also log to file (which must exist!)
</code>

===== logrotate =====

''logrotate'' allows automatic rotation, compression, removal, and mailing of traditional log files. Each log file may be handled daily, weekly, monthly, or when it grows too large. It is started regularly (typically daily) via a cron job and should help to avoid running out of disk space due to old log files.

Which logs should be rotated is configured in ''/etc/logratate.conf'' (or separate files in ''/etc/logrotate.d'')

An example taken from ''/etc/logrotate.d/rsyslog'':
<code>
/var/log/debug
/var/log/messages
{
        rotate 4
        weekly
        missingok
        notifempty
        compress
        delaycompress
        sharedscripts
        postrotate
                reload rsyslog >/dev/null 2>&1 || true
        endscript
}
</code>

===== syslogd =====

''syslogd'' is part of ''systemd'' and changes the way logging works drastically. ''syslogd'' collects logs from the early boot process, the kernel, the initrd, and even application standard error and out. Configuration is done in ''/etc/systemd/journald.conf''.

All logs are stored in binary format (typically below ''/var/log/journal'') and can be viewed in syslog-style or exported in other formats with ''journalctl'':
<code bash>
journalctl --utc              # view logs with UTC timestamps
journalctl -k                 # view kernel messages (similar to dmesg)
journalctl --since "2015-01-10" --until "2015-01-11 03:00"
journalctl -b -o json-pretty  # get log since last boot in JSON format
journalctl -f                 # print logs and follow (like tail -f)
</code>

[[https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-view-and-manipulate-systemd-logs|Check out this tutorial for more.]]
====== Process Management ======


===== Process Information =====

Via procfs exhaustive information about each process is available in ''/proc/<process_number>''. For must use cases these programs are commonly used:

== ps ==
''ps'' extracts relevant information about current processes. It is more useful in scripts than the interactive programs ''top'' or ''htop'', which provide about the same information.
<code bash>
ps                     # print all processes with same effective current user in current terminal
ps axu                 # print all processes (a=for all users, x=also if no tty, u=user format)
ps -l ax               # print all processes in long format (with e.g. nice value)
ps -u user -U user u   # print all processes for user in user format
ps axu --sort %mem     # print all proceses sorted by memory consumptin (cpu: %cpu)
</code>

Process trees give a good overview of parent-child relationships:
<code bash>
ps axjf                # print process tree
pstree                 # print even nicer process tree
</code>

== pgrep ==
This is a good replacement for using first ''ps'' and then piping its output into ''grep''.

<code bash>
pgrep java             # prints all ids of processes with java in its name
pgrep -f java -u user  # prints all ids of processes with java in its full command line and belong to user
</code>
===== Starting & Stopping Processes =====
Bash normaly executes commands in the foreground. Commands ending with an ampersand (&) are started in the background. A currently executed foreground-process can be stopped (SIGSTOP) by pressing CTRL-z.

Apart from that the [[http://web.mit.edu/gnu/doc/html/features_5.html|following bash builtins]] can be used for execution control:

<code bash>
jobs                   # list jobs (and their numbers)
bg                     # put last process into background (highest nr in 'jobs')
bg <nr>                # put process into background (same as start with &)
fg                     # put last process into foreground (highest nr in 'jobs')
fg <nr>                # put process into foreground
kill -s TERM %<nr>     # send signal to job (-s 15 and -s TERM are equivalent)
</code>
In addition to the bash builtins ''kill'' is also a program (see ''type -a kill''). It can only work with process ids and not job ids. Actually kill can send any signal to a process. The most used signals are
  * SIGTERM (15): politely ask a process to terminate gracefully and clean up all resources (default for ''kill'', ''killall'')
  * SIGINT (2): ask a process to terminage (ungracefully) - often used for programs running from the terminal (CTRL-c)
  * SIGQUIT (3): same as SIGQUIT, but also writes a core dump on some systems (CTRL-\)
  * SIGKILL (9): force-kill unresponsive processes - only use if less drastic signals had no effect
  * SIGSTOP (17,19,23): stop (pause) a process (CTRL-z)
  * SIGHUP (1): tell a program to re-read its configuration files

In cases where the job or process id is unknown or multiple instances of the same program must be killed (or supplied with a signal) these two (more or less equivalent) programs can be used:
<code>
killall -s KILL command  # kill all processes of command (exact name required, or use -r to interpret as regex)
pgrep -l under           # list all processes whose *program* contains the string under, e.g. thunderbird
pkill under              # send all matching processes SIGTERM
</code>

== nohup ==
''nohup'' starts commands and protects them from signals sent when the shell it was created within is terminated (e.g. when the remote session on a server is closed when you log out). Output is automatically redirected to nohup.out.
In combination with the ampersand to execute a command in the background this is quite useful for long-running tasks on remote machines (when not using 'screen' or 'byobu'):
<code bash>
nohup command            # start command detached from console
nohup command &          # start command detached from console (in background)

</code>


===== Process Priorities =====
nice values represent the niceness of a process from from -20 (most favorable scheduling) over 0 (default) to 19 (least favorable).

<code bash>
nice -n -20 program      #start program with a nice-value of -20 (values < 0 may require root!)
nice --20 program        #same result, but not very readable
renice -n -20 <pid>      #give a process a new nice-value
</code>

====== Scheduling Execution of Jobs ======

===== One-Shot Jobs =====

''at'' and ''batch'' schedule the one-time execution of a command at a given time (''at'') or when the system load is low enough (''batch''). The job is read from stdin (or a file provided with -f).

<code bash>
at <time> (<day>)  # interactively schedule execution for a given time
batch              # interactively schedule execution when system load allows (same as at -b)
atq                # view queue of user (root sees all jobs)
atrm <jobid>       # remove scheduled job
</code>

== Schedule a job ==

Start ''at'' with a time (and optional day) and interactively type commands (may span several lines) and then Ctrl+d when finished.

Various time and date formats can be used, e.g.

<code bash>
at 18:10 2014-10-13
at midnight
</code>

To see the contents of a job, which are stored in ''/var/spool/cron/atjobs'', use
<code bash>
at -c <jobid>
</code>

== Permissions ==

With ''/etc/at.allow'' and ''/etc/at.deny'' you can configure who is allowed to schedule jobs. Add one user per line (without whitespace).

===== Regular Jobs (Cron Jobs) =====

''cron'' is a deamon used to schedule the regular execution of programs / scripts / commands. A scheduled task is typically called (cron) job.

The maximum time resolution is minutes, a job can therefore be executed at maximum once a minute.

Cron jobs can be configured in two ways.

==== Jobs in /etc/cron.* ====

The system-wide approach is to put an executable (script) in one of these self-explanatory directories:

<code bash>
/etc/cron.hourly
/etc/cron.daily
/etc/cron.weekly
/etc/cron.monthly
</code>

==== /etc/crontab ====

The second way is via crontrab-entries into the global crontab file ''/etc/crontab'' with the following format (must end with a newline!):
<code>
  0   2   12  *   *   user /usr/bin/false
# ┬   ┬   ┬   ┬   ┬   ┬    ┬
# │   │   │   │   |   │    └ command
# │   │   │   │   |   └───── user executing the command
# │   │   │   │   └───────── day of week (0 - 7) (0 to 7 are Sunday to Sunday, or use names)
# │   │   │   └───────────── month (1 - 12)
# │   │   └───────────────── day of month (1 - 31)
# │   └───────────────────── hour (0 - 23)
# └───────────────────────── min (0 - 59)
</code>

==== More crontab ====
Also possible, but not encouraged in order to not put cron jobs into too many different places and overcomplicate administration, are the following options:

Put files in the same format as ''/etc/crontab'' into ''/etc/cron.d''.

Or use the program ''crontab'', which edits a special crontab file for each user and validates the file at each save. The cron job will be executed by the respective user (with the according rights).

<code bash>
crontab -l     # view (list) crontab file for current user
crontab -e     # edit crontab file  for current user
crontab -r     # remove crontab file for current user
</code>

The crontab files created by the command ''crontab'' for each user reside in ''/var/spool/cron/crontabs'' (and omit the user-column!).

== Intervals, Repetitions,... ==

Minutes, days,... can be specified as follows
^ format ^ example ^ further description |
| single value | 1 | |
| comma-separated list | 0,3,6 | |
| dash-separated range | 0-3 | |
| */<nr> | */5 (e.g. every 5 minutes starting at 0: 0,5,10,...) | every <nr> beginning at 0 |
| <start>/<nr> | 2/5 ( e.g. every 5 minutes starting at 2: 2,7,12,...) | every <nr> beginning at <start> |

**Gotcha:** when specifying e.g. 2/5 in minutes and the system boots at 18:03 the first job is executed at 19:03 because 18:02 was missed!


== Special Strings == 

The following special strings can be used instead of the first five fields of a row:

^ string   ^ replaces  ^ meaning    ^
| @reboot  | -         |run once, at startup |
| @yearly, @annually | 0 0 1 1 * | run every first minute of a year |
| @monthly | 0 0 1 * * | run every first minute of a month |
| @weekly  | 0 0 * * 0 | run every first minute of a week (on Sunday!) |
| @daily, @midnight | 0 0 * * * | run every first minute of a day |
| @hourly  | 0 * * * * | run every first minute of an hour | 

== Permissions == 

To configure which user can use ''crontab'' add respective entries (one user name per line) to
<code bash>
/etc/cron.allow # takes precedence
/etc/cron.deny
</code>

== Further Resources ==
[[https://help.ubuntu.com/community/CronHowto|Ubuntu Tutorial]], [[http://www.pantz.org/software/cron/croninfo.html|In-depth explanations]]
==== Disable a cron job ====

To disable a cron job in ''/etc/cron.*'' either remove the executable flag from the script / program (''chmod -x script'') or move the script out of the folder.

Jobs in crontab are disabled by prepending the line with a hash (#).

==== Gotchas ====

Cron passes only a minimal set of environment variables to your jobs. E.g. the ''PATH'' may be different than what you expect, things set in ''.bashrc'' may not be present,...

''[[http://askubuntu.com/questions/23009/reasons-why-crontab-does-not-work|More info.]]'' ''[[https://help.ubuntu.com/community/CronHowto|Ubuntu CronHowTo]]''

==== Anachron ====

For computers not running 24/7 or in cases when missed jobs due to downtime must be run as soon as the server is up again use ''anacron''.

Jobs for anacron are specified in ''/etc/anacrontab'', which only root can edit (there is no equivalent to the command ''crontab'' usable by every user as for regular cron jobs) 

<code>
# environment variables
SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
RANDOM_DELAY=30
START_HOURS_RANGE=3-22

# jobs
  1   15  backup  /usr/local/bin/my_backup.sh
# ┬   ┬   ┬       ┬   
# │   │   │       └── command (may span several lines if line is terminated with '\'
# │   │   └────────── job-identifier string (should be unique)
# │   └────────────── delay in minutes (e.g. after system startup)
# └────────────────── period in days (>=1) or @period_name (e.g. @daily,..)
</code>

Two environment variables can be used by putting them before the job definition in ''/etc/anacrontab'':
  * START_HOURS_RANGE=3-22 (job may be run between 03:00 and 22:00)
  * RANDOM_DELAY=30 (a random delay up to 30 added to 'delay in minutes')

The timestamp (YYYYMMDD) of the last execution for each job is stored in ''/var/spool/anacron/<job-identifier>''.

====== X Server ======

The X Window System (X11) is a windowing system that handles **user data input** (e.g. keyboard) and **graphical output**. It uses a client server model, where one server can handle several clients.

A complete graphical environment consists of four layers:
  - Desktop Environment Manager: collection of programs designed for a good user experience, look&feel,.. (KDE, Gnome,..)
  - Window Manager: draw and manager windows (KWin, Metaciy, Enlightenment,..)
  - Display Manager: handle user login and start window manager (lightdm, kdm, gdm, xdm,..)
  - Display Server: interface to hardware (X-Server, wayland,..)

X can be started with the command ''X''.
===== Configuration with xorg.conf =====

The central configuration file to configure all aspects of X11 is ''/etc/X11/xorg.conf'', although in 2015 this file is seldomly used. Instead auto-configuration (and hotplugging) through ''xrandr'', HAL, udev,... is used.

The file is structured in sections, the most important being:
  * Monitor: properties of physical monitors (e.g. monitor modelines)
  * Screen: configuration of screens (that may span several monitors)
  * Device: configuration of video card 
  * Files: locations of fonts (FontPath) or modules (ModulePath)
  * InputDevice: configuration of keyboard, mouse, touchpad,..

===== Parameters for Starting Programs =====

==== geometry ====

The parameter ''-geometry'' defines width, height, x offset (from left side) and y offset (from top) in that order. The unit for width and height depend on the program (e.g. pixels or characters)

<code bash>
xterm -geometry 120x20+100+100  # 120 characters wide, 20 lines tall, 100 pixels offset to top left corner
xterm -geometry 120x20-0-0      # same as above, but in the right lower corner of the screen
</code>

==== display ====

With ''-display'' the display where the started program should be displayed is chosen. Because of the client server architecture of X it is possible to run a program one computer and display it on another.

The syntax is: ''hostname:XServer.Display'', but hostname and display can be omitted.

<code bash>
xeyes -display :0             # display xeyes on the first display of the first local X-Server
xeyes -display remotehost:1:2 # display xeyes on the third display of the second X-Server on remotehost
</code>

The default display is stored in the environment variable ''DISPLAY''.
<code bash>
export DISPLAY=":0"
</code>

===== X Over Network =====

==== Display on Remote Host ====

As mentioned before the ''DISPLAY'' variable can contain other hosts - meaning the window will be shown on a different host.

To configure an X-Server to receive connections via TCP/IP:
  * X must not be started with ''-nolisten tcp'' (as is the case in e.g. Ubuntu: see ''/etc/X11/xinit/xserverrc'')
  * the display manager must be configured accordingly (e.g. lightdm by default disables tcp listening, it must be turned on by adding ''xserver-allow-tcp=true'' in ''/etc/lightdm/lightdm.conf''
  * the remote host must have been whitelisted with ''xhost''

<code bash>
xhost +             # allow connections from everywhere
xhost -             # disallow connections from non-listed IPs
xhost +localhost    # allow TCP/IP connections from localhost
xhost -localhost    # disallow TCP/IP connections from localhost
xhost               # show current status
</code>

If in doubt check if X11 is listening on port 6000 (''nmap -p 6000 localhost'')

==== X over SSH ====

When connected to a remote machine over SSH it is often useful to redirect the graphical output to the local machine. This can be done with the parameter ''-X''.

<code bash>
ssh -X remotehost
</code>

However, the remote host must allow this: ''X11Forwarding yes'' must specified in ''/etc/ssh/sshd_config''. [[http://unix.stackexchange.com/a/12772|(more explanations)]]

==== Terminal Server ====

The X display manager control protocol (XDMCP) is used by display managers (e.g. xdm) to look for X servers where the client can log in. However XDMCP is insecure (no encryption) and slow (no compression) - alternatives are recommended. ([[https://wiki.ubuntu.com/xdmcp|Ubuntu documentation]])
===== Fonts =====

On Linux systems fonts typically can be found in ''/usr/share/fonts''. In ''/etc/fonts/fonts.conf'' these locations are defined.

Fonts must reside in one of the font directories and can only be used after ''mkfontdir'' and ''mkfontscale'' have been used on the respective directory. These tools create the helper files ''fonts.dir'' and ''fonts.scale''.

==== X-Font-Server ====

With the X-Font-Server ''xfs'' a central server can be used to maintain a collection of fonts served to its clients. This server can be configured in ''/etc/X11/xfs.conf''.

Clients must add an entry for the server into ''/etc/X11/xorg.conf'':
<code>
Section "Files"
    FontPath "tcp/192.168.0.2:7100"
EndSection
</code>




===== Accessibility =====

To provide better accessibility for the physically disabled you can use tools like on-screen keyboards, text to speech (e.g. [[https://wiki.gnome.org/Projects/Orca|Orca]], emacspeak), high contrast desktop themes with large fonts, magnifiers, output for Braille displays, or adapted keyboard sensitivity.

Most of the use cases are covered by various programs of your favourite desktop environment, but for adapting the keyboard we can use core functionality of X: AccessX.

With ''xkbset'' keyboard-related features can be turned off (with ''-'') or on (without ''-''):
<code bash>
xkbset -repeatkeys        # pressing a key once (no matter how long) counts as one key press
xkbset slowkeys 500       # a key counts as pressed after 500ms 
xkbset bouncekeys 500     # pressing a key twice within 500ms only counts as one key press
</code>

===== Tools =====

Some useful tools in an X environment:
<code bash>
xwininfo         # prints details about a program after klicking it (or supplying its id)
xkill            # kills a program after klicking its window (or supplying its id)
xdpyinfo         # prints information about the X server
xrandr           # configure the X-server on the fly
</code>
====== Networking ======


Networking is based on the [[http://en.wikipedia.org/wiki/OSI_model|ISO/OSI model]] consisting of seven layers.
  - Physical (network hardware)
  - Data Link (MAC)
  - Network (IP)
  - Transport (TCP, UDP)
  - Session (sockets)
  - Presentation (SSL)
  - Application (HTTP)

===== IP, TCP, UDP =====

TCP (connection-based) and UDP (connectionless) are very common transport protocols based on the IP network protocol. Their differences are [[http://www.diffen.com/difference/TCP_vs_UDP|explained here]]

The internet protocol (IP) is currently in use in v4 and v6, see [[http://core0.staticworld.net/images/article/2014/10/irishtelecom-100478043-large.idge.jpg|this infographic]] for detailed comparison and the most important facts here:

^               ^ IPv4                                       ^ IPv6      ^
| address bits  | 32                                         | 128      |
| addresses     | ~4.3 billion (''10^9'')                        | ~340 undecillion (''10^36'') |
| notation      | dot-separated numbers (8 bits: 0-255) | colon-separated hex (16 bits: 0-FFFF) |
| localhost     | 127.0.0.1                                  | ::1                                              |
| full example  | 62.218.164.154                             | 2001:0db8:0000:08d3:0000:8a2e:0070:7344          |

For IPv6 addresses leading zeros can be omitted (000A = A or 0000 = 0) and one group of consecutive zeros can be shortened with a double-colon (::).
IPv4 requires [[http://en.wikipedia.org/wiki/Network_address_translation|network address translation (NAT)]] to map several hosts in a local network to one public IP. NAT is no longer required with IPv6.

==== TCP/UPD Port Numbers ====

Port numbers (16 bit with 65536 possible ports) map to single services behind an IP. Conventions for port assignments are managed by IANA (Internet Assigned Numbers Authority) who provide an [[http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml|up-to-date list of known ports]]. On Linux systems ''/etc/services'' contains the known ports.

The port range of 0 - 65535 is split into three regions:
  - 0 - 1023: system / well-known ports (ports registered with IANA, root required to open a port)
  - 1014 - 49151: user / registered ports (ports registered with IANA, a normal user can open a port)
  - 49152 - 65535: dynamic / private ports (no rules)

=== Common Ports ===

^ Port  ^ Usage         ^ Full Name                                            ^
| 20    | ftp-data      | file transfer protocol                               |
| 21    | ftp-control   |                                                      |
| 22    | ssh           | secure shell                                         |
| 23    | telnet        |                                                      |
| 53    | domain        | domain name service (dns)                            |
| 80    | http          | hypertext transfer protocol                          |
| 123   | ntp           | network time protocol                                |
| 139   | netbios-ssn   | network basic input output system (session service)  |
| 161   | snmp          | simple network management protocol                   |
| 162   | snmp traps    |                                                      |
| 389   | ldap          | lightweight directory access protocol                |
| 443   | https         |                                                      |
| 514   | rsh & syslog  | remote shell & syslog                                |
| 636   | ldaps         |                                                      |

=== Email ===
^ Port  ^ Usage  ^ Full Name                         ^
| 25    | smtp   | simple mail transfer protocol     |
| 110   | pop3   | post office protocol              |
| 143   | imap4  | internet message access protocol  |
| 465   | ssmtp  |                                   |
| 993   | imaps  |                                   |
| 995   | pop3s  |                                   |


==== IPv4 Netmasks ====

IP4v netmasks are 32 bits long. They consist of a consecutive number of set bits and then only unset bits. The Classless Inter Domain Routing (CIDR) notation gives the number of set bits. The number of hosts in a network is defined as 2^nr-of-0-bits minus 2. The lowest (network address) and highest (broadcast address) addresses have to be deducted.

List of class C networks (with only the last 8 bit block available):

^ Dot Notation ^ CIDR Notation ^ Bitmask Notation ^ Number of Hosts ^
| 255.255.255.0 | /24 | 11111111.11111111.11111111.00000000 | 254 |
| 255.255.255.128 | /25 | 11111111.11111111.11111111.10000000 | 126 |
| 255.255.255.192 | /26 | 11111111.11111111.11111111.11000000 | 62 |
| 255.255.255.224 | /27 | 11111111.11111111.11111111.11100000 | 30 |
| 255.255.255.240 | /28 | 11111111.11111111.11111111.11110000 | 14 |
| 255.255.255.248 | /29 | 11111111.11111111.11111111.11111000 | 6 |
| 255.255.255.252 | /30 | 11111111.11111111.11111111.11111100 | 2 |
| 255.255.255.255 | /32 | 11111111.11111111.11111111.11111111 | 1 |

Note, that /31 is missing because it would only consist of a network and a broadcast address but no host. /32 is a special case identifying a single address.
The n
===== Network Configuration =====

Physical network interfaces were typically named ''ethN'' and ''wlanN''. For virtual interfaces (several interfaces over one physical interface) a colon is added, e.g. ''eth0:0''.

Modern systems using systemd use [[https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames|predictable network interface names]] in the form ''en'' (ethernet), ''wl'' (WLAN), or ''ww'' (WWAN), e.g. ''enp0s25''. The advantage of this new naming schema is, that the names will not change on reboots, which could happen with the old scheme.

Inspection of the current state and temporary configuration can be done with ''ifconfig'', ''route'' and ''ip''.

==== Dynamically Configure IP, Netmask, Broadcast ====

<code bash>
ifconfig -a      # view all interfaces (also disabled ones)
netstat -i -e    # same..
</code>

Basic configuration:

<code bash>
ifconfig eth0 192.168.0.1  # minimum
ifconfig eth0 192.168.0.1 netmask 255.255.255.240 broadcast 192.168.0.16
</code>

Turning interfaces on and off:

<code bash>
ifconfig eth0 up
ifconfig eth0 down
</code>

Virtual interfaces can be simply created if the underlying physical interface exists
<code bash>
ifconfig eth0:0 192.168.0.1  # create
ifconfig eth0:0 down         # disable / remove
</code>

==== Dynamically Configure Gateway ====

The gateway address is the address packets not destined for our current subnet are sent (and then e.g. sent into the internet through a router). It is set via ''route'' in the kernel's IP routing table.

<code bash>
route            # view current routing table 
route -n         # same, but with numeric IP addresses instead of names
</code>

Set a default route (to a gatewy) for a network
<code bash>
route add default gw 10.0.0.138
</code>

Add route for a specific network (network address and netmask must match!) and a specific device:
<code bash>
route add -net 10.0.1.0 netmask 255.255.255.0 gw 10.0.0.138 dev eth0 
</code>

For removing routes the exact same command has to be executed, but with ''add'' replaced by ''del'':
<code bash>
route del -net 10.0.1.0 netmask 255.255.255.0 gw 10.0.0.138 dev eth0
</code>

==== Statically Configure the Network ====

To permanently configure the network configuration (also after a reboot) use ''/etc/network/interfaces''. All interfaces configured here can be en/disabled with ''ifup'' and ''ifdown''.

Use DHCP:
<code bash>
auto eth0
iface eth0 inet dhcp
</code>

Manual configuration:
<code bash>
auto eth0
iface eth0 inet static
address 192.168.0.100
netmask 255.255.255.0
gateway 192.168.0.1
</code>

Note, that since at least Ubuntu 12.04 desktop versions use [[https://help.ubuntu.com/community/NetworkManager|NetworkManager]] instead of ''/etc/network/interfaces''. Use the respective GUI tools or ''nmcli'' and ''nm-tool'' for configuration.

==== Name Resolution (Static) ====

Simple static name resolution can be configured in ''/etc/hosts'', which contains a simple name to IP mapping.

Usage of the Domain Name Service (DNS) is configured in ''/etc/resolv.conf''. The keyword ''nameserver'' specifies DNS servers:

<code bash>
nameserver 8.8.8.8
nameserver 8.8.4.4
</code>

[[http://unix.stackexchange.com/questions/128220/how-do-i-set-my-dns-on-ubuntu-14-04|In Ubuntu 14.04]] or later you should edit ''/etc/resolvconf/resolv.conf.d/head'' and then run ''resolvconf -u'', since ''/etc/resolv.conf'' is generated.

In ''/etc/nsswitch.conf'' you can configure if and in which order local files (''/etc/hosts'') and DNS should be used:
<code bash>
hosts:    files dns
networks: files dns
</code>


DNS (reverse) lookups can be done with ''host'' or the 'domain information groper' ''dig'' (''nslookup'' is obsolete due to security issues!).

<code bash>
host -a <hostname>   # look up DNS entry for a hostname
dig <hostname>       # same, but more verbose (and more options available)
</code>

===== Network Troubleshooting =====

==== Check State of Network Hardware / Connection ====
<code bash>
ethtool              # query or control network driver and hardware settings
nm-tool              # report NetworkManager state and devices
</code>

==== Check Routes ====
<code bash>
tracepath <host>     # trace the path to a network host (which servers / hops are on the way)
tracepath6 <host>    # for IPv6
traceroute <host>    # same as tracepath, but requires root
traceroute6 <host>   # for IPv6
</code>

==== Check Open Ports & Connections ====

''netstat'' can provide very detailed information about open connections and statistics:

<code bash>
netstat -a           # print open & listening internet connections and Unix domain sockets
netstat -a -t        # print only tcp connections (-u for udp) 
netstat -s           # print statistics about e.g. nr of received packets
netstat -r           # print the kernel routing table
</code>

''nmap'' is an advanced port scanner ([[http://www.cyberciti.biz/networking/nmap-command-examples-tutorials/|see some examples here]]). A simple use case is to check which services are available / ports are open on a server:
<code bash>
nmap localhost       # print open ports on current machine (only a selection of common ports)
nmap -p 1-9999 host  # print open ports on a host (with a defined port range)
nmap -A -T4 host     # print open ports with enabled version detection (-A) and don't wait too long (-T4)
</code>


[[https://help.ubuntu.com/14.04/serverguide/network-configuration.html|Advanced guide for Ubuntu network configuration]]

===== Network Services with (x)inetd =====

The deprecated ''inetd'' and the modern (and icompatible) ''xinetd'' are super-daemons that listen for incoming requests on several ports and start the according services only on demand. This can be useful to save resources by running rarely used services only when needed.

''inetd'' was configured via one entry for each service in ''/etc/inetd.conf'' (where the service name should be present in ''/etc/services'' or a port number):
<code>
# service     type     protocol  wait    user    programm            arguments
telnet        stream   tcp       nowait  root    /usr/sbin/telnetd   telnetd -a
pop3          stream   tcp       nowait  root    /usr/sbin/tcpd      ipop3d
3000          stream   tcp       nowait  nobody  /bin/echo           echo Hello World
</code>

''xinetd'' provides additional features such as access control mechanisms (e.g. with TCP Wrapper ACLs), extensive logging capabilities, the ability to make services available based on time or the maximum number of started services (to counter DoS attacks). It is configured via ''/etc/xinetd.conf'' and one file for each service in ''/etc/xinetd.d/''.
An example configuration for the TCP version of daytime that prints the current date and time:
<code>
service daytime
{
        disable         = yes
        type            = INTERNAL
        id              = daytime-stream
        socket_type     = stream
        protocol        = tcp
        user            = root
        wait            = no
}
</code>

==== TCP wrapper ====

''tcpd'' is a TCP wrapper which first performs security checks and then starts a program (see pop3 service in the inetd-example above). ''tcpd'' checks if the incoming request is allowed through the settings in ''/etc/hosts.allow'' and ''/etc/hosts.deny''. The check if access is granted works as follows: first allow is checked. If not explicitly allowed it is checked if it is explicitly denied. If not it is allowed. The lines in each are processed in order of appearance and search terminates when a match is found (so put more specific rules first).

Syntax: ''daemon : client [:option1:option2:...]'' (see ''man hosts_access''), here are some simple examples:
<code>
vsftpd: 192.168.1.,.abc.com
sshd: ALL EXCEPT 192.168.0.15
ALL: ALL                 
</code>

====== Email ======
Email basically works as follows.

The sender uses a **Mail User Agent (MUA)**, or email client, to send the message through one or more **Mail Transfer Agents (MTA)**, the last of which will hand it off to a **Mail Delivery Agent (MDA)** for delivery to the recipient's mailbox, from which it will be retrieved by the recipient's email client, usually via a POP3 or IMAP server. 

Commonly used programs are:
  * MUA: Mozilla Thunderbird, Microsoft Outlook,..
  * MTA: sendmail (the original), postfix, exim, postmaster, qmail, smail
  * MDA: procmail, maildrop

==== Set up a local email Server ====

<code bash>
apt-get install postfix
dpkg-reconfigure postfix  (basically edits /etc/postfix/main.cf)
</code>

No matter which MTA is used, they all share:
  * ''sendmail'' as command to send emails
  * ''/etc/aliases''

== aliases ==

With ''/etc/aliases'' emails to local recipients can be redirected to one or multiple recipients.

The format for each line is:
<code>
#name: value1, value2, ...
postmaster: alice
root: alice, bob
</code>

After changes to ''/etc/aliases'' the command ''newaliases'' or ''sendmail -bi'' must be called.
It updates the alias-database, which is used by sendmail et al instead of the raw text file.

Each user can create redirections for only his/her account in ''~/.forward'' using the same format.

More in the [[https://help.ubuntu.com/14.04/serverguide/postfix.html|Ubuntu server guide]]


==== Send Emails ====

An email consists of a header and a body, separated by a new line. A very simple Email:
<code>
Subject: Hello World

This is the body of my first email.
</code>

To send it the command ''sendmail'' can be used (no matter which MTA is actually used):
<code bash>
sendmail recipient@example.com < email.txt
sendmail -bm -f sender@example.com recipient@example.com < email.txt
</code>
==== Read Emails ====

Incoming emails are stored in the mail spool in ''/var/mail/username''. The spool can e.g. be inspected with the command line tool ''mail''.

Typically MUAs move emails from the spool to a different location. MUAs that support mail spools are e.g. mutt and pine (command line) or Mozilla Thunderbird (create a 'Unix Mailspool (Movemail)' account).

====== Printing ======

TODO FIXME


====== Verify Suspect Files / Packages ======
find package of suspect file
<code>
dpkg -S <filename>
</code>

basic check:
<code>
debsums <package_name>
</code>

a little more paranoid check: 
<code>
aptitude download <package_name>
ar x <package.deb>
tar xfvz control.tar.gz
grep $(md5sum <filename>) md5sums
</code>
Small heaps of code

User Tools

Site Tools

Page Tools
