Jörgs LinuxPages

Save System Info

saveconfig is a handy script to get a copy of the most important configuration files in one run - for backup, documentation and for the preparation of a system update or a fresh install on another machine. It saves the most important configuration files as tar-archive to any directory and provides a textual listing of some of those files. - Note that some of the commands are only accessible as root.

Download the saveconfig script for deb-based systems (Debian, Ubuntu, ...) and for rpm-based systems (Fedora, RedHat, SuSE, ...).

Accessing the console

To change from the graphical screen to a text console (aka "terminal"), use Alt-Ctrl-F1 (and the other F-keys for more). With Alt-Ctrl-F10 you get the system messages - if you leave your computer unattended for a long time, it is a good idea to switch to this console before.

To access (grab) the image of a text console, use the device files vcsn. You can access them as user root.

System log on console 9

Usually, system logs are written to /var/log/messages. If you want a "live copy" of these messages simultaneously on console 9, simply edit the file /etc/syslog.conf and duplicate the line pointing to /var/log/messages as follows:

*.*;mail.none;news.none         -/var/log/messages
*.*;mail.none;news.none         /dev/tty9

After running rcsyslog restart, system messages will appear also on console 9 (reachable with Alt-Ctrl-F9).

(Thanks to Romeo Benzoni in ch.comp.os.linux!)

The SysRq key

To make use of the kernel SYSRQ key, set ENABLE_SYSRQ=yes in /etc/sysctl (on older systems: /etc/rc.config). If you press the key combination ALT + SysRq (aka PrtScr) plus one of the following keys, this will do ...

• r turns off keyboard raw mode and sets it to XLATE.
• k kills all programs on the current virtual console - aka "secure access key", as it will kill trojans.
• b will immediately reboot the system without (!) syncing or unmounting your disks.
• o will shut your system off (if configured and supported).
• s will attempt to sync all mounted filesystems.
• u will attempt to remount all mounted filesystems read-only.
• e sends a SIGTERM to all processes, except for init.
• i sends a SIGKILL to all processes, except for init.

"Soft" NFS mounts

If you use Linux in a network, you will probably use NFS to mount remote directories. Timing problems may occur if the NFS server is not available at the NFS client's boot time, or if the server goes offline while the client is still connected. To allow "soft" mounts in /etc/fstab, simply add the options bg,hard,intr. See the manpage to mount for details.

Corrupt filesystem?

If ever you happen to come across a corrupt filesystem (e.g. you installed a new harddisk, but forgot to update /etc/fstab and now one of your partitions became a swap partition ;-), the program e2fsck if of use.

However, upon running e2fsck /dev/hdX you may run into trouble if the superblock is defective. In this case, you may specify an alternative (backup) superblock, e.g. e2fsck -b 8192 /dev/hdX.

To find out where these backup copies are stored, just run mke2fs -n /dev/hdX ... this command causes mke2fs to display what it would do if it were to create a filesystem, but without actually doing it. In other words, it shows the location of the backup copies of the superblock! All you need to do then is to re-run e2fsck with the -b option and an appropriate superblock location.

SCSI reset

Sometimes I have to abort a CD burning process, or I forgot to switch on one of the devices on the SCSI bus. Now, every reboot is a reboot too much, so here is a way to add a SCSI device when the system is already running.

Removal (necessary for "screwed-up" devices):

echo "scsi remove-single-device a b c d" > /proc/scsi/scsi

Adding a new device:

echo "scsi add-single-device a b c d" > /proc/scsi/scsi

where a is the hostadapter id (first one being 0), b is the SCSI channel on hostadapter (first one being 0), c is the device ID and d the LUN number (first one being 0). With a single SCSI card, usually all you have to care about is the device ID c, the rest is 0.

In the case of my CDRW, which is the 6th device on the first SCSI bus, this is as simple as

echo "scsi remove-single-device 0 0 6 0" > /proc/scsi/scsi
echo "scsi add-single-device 0 0 6 0" > /proc/scsi/scsi

Many thanks to Patrick Fuerlinger in ch.comp.os.linux!
Note: SuSE Linux comes with a script rescan-scsi-bus.sh for that purpose.

Adding SCSI devices

The same command that was mentioned in the previous section allows to switch on a SCSI device after the computer has started - as an example, when I want to do a backup but forgot to switch the tape drive on at boot time:

echo "scsi add-single-device 0 0 4 0" > /proc/scsi/scsi

It does not hurt if you get the wrong device ID. If you chose the right one, the command cat /proc/scsi/scsi should list the new device.

Please keep in mind that the SCSI bus is not hotpluggable - the device must already be present in the SCSI chain, it just does not matter if it was switched on.

Setting System Time
via Internet

If you use a dial-up connection, timehosts is a simple script to synchronise both the system time and the hardware clock with a public time standard via Internet. - You will need to adjust the ntp server according to your timezone.

Download the timehosts script. Now supports also server on the local network (option '-l').

Keyboard layout

If you find yourselves in a shell with a "strange" keyboard layout and want to switch to your usual layout (here, Swiss French):

setxkbmap ch fr

Some PDF tricks

If you want to remove the cover page from a document you received (e.g. a fax):

convert -verbose -page A4 -compress Zip multipagefax.tif[1-4] fax.pdf

If you have a multi-page pdf file that you would like to OCR, you can split it into individual TIFF files:

pdftopbm -r 220 mytxt.pdf mytxt
for i in mytxt-0* ; do convert -verbose $i `echo $i|sed 's/pbm/tif/g;s/000//g'` ; done

If you have a pdf file that includes way-too-detailed images (such as PDF from badly programmed websites where the so-called "thumbnails" were actually full-size images, squeezed into the small space by abusing the height and width attributes), you can rewrite these files with GhostScript. We often use the following bash command to significantly shrink the file size while preserving all visible information, including texts:

for FILE in big*.pdf; do 
  gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/default -dNOPAUSE -dQUIET -dBATCH \
  -dDetectDuplicateImages -dNOTRANSPARENCY -dCompressFonts=true -sOutputFile=x.pdf $FILE \
  && mv x.pdf $FILE 
done

Find files

You certainly know this problem: Somewhere in your directory tree is a file, but all you remember is that is contains the words "what text". To find this file, just issue the following command:

find . -type f -print0 | xargs -0 grep -li "what text"

The find command finds all "normal" files and feeds a list of them to the pipe. xargs takes the piped list and makes it into an argument list for grep. The -i option makes grep ignore case and -l is used to output a list of unique filenames which match. The options -0 to xargs (and -print0 instead of print to find) help to deal with whitespace in filenames.

(Inspired by a tip found at www.linuxplanet.com).

Note that this will not work with files such as PDF that encode their content. Here, you may need to use dedicated tools such as pdfgrep:

pdfgrep "whatever text" file*pdf

Find files that were modified

To find all PDF files that were modified in the past 24 h:

find . -type f -name "*.pdf" -mtime -1

To find all HTML files that were modified in the past 30 min:

find . -type f -name "*.html" -mmin -30

Understand config files

Here's a quick one-liner to print all lines in a config file that are neither commented out (starting with #) nor blank:

cat /path/to/file.conf | grep -v ^# | grep -v ^$

The grep -v ^# command prints all lines that are not starting with the comment sign. The output is piped into grep -v ^$ which prints all lines that are not empty.

Search-and-replace with files

There are many ways to replace a certain text in a number of files. Here is a one-liner in Perl:

perl -e 's/old text/new text/gi' -p -i.bak *.html

If this did no go as expected, we cam rename the .bak files back to the original:

rename -n -v 's/\.bak$//' *.bak

... and even recursive:

find .  -type f -name "*.bak" -exec rename -v -f  's/\.bak$//' {} \;

Another search-and-replace example, using find and sed to remove an alternate stylesheet specification from all htm(l) files below a given directory. This one involves escaping the slashes and dots:

find . -type f -name "*.htm*" -exec sed -i '/css\/classic\.css/d' {} \;

Replacing double question marks h??s by h?s requires a lot of escaping:

find .  -type f -name "*.htm*" -print | xargs perl -p -i.bak -e 's/h\?\?s/h\?s/g;'

Search-and-rename directories

I needed to rename a number of directories in-place, from archiv to archive. The following code did the job:

find . -name "archiv" -type d -execdir mv {} archive \;

Remove whitespace in filenames

I often have to deal with files that contain spaces in the filename, such as "My File". Although Linux can deal with such names, I prefer to rename them so that the names do not contain any whitespace nor other special characters. The following one-liner (in bash) replaces the space character by underscore, so My File becomes My_File:

for i in *\ *; do mv "$i" "`echo $i | sed 's/ /_/g'`";  done

While we are at it, let's transcribe all uppercase characters to lowercase (My File becomes my_file):

for x in *; do mv "$x" `echo $x | tr "A-Z " "a-z_"`; done;

Applying this through a whole directory structure, with all files and directories, starting at the current working directory, with find:

find . -exec bash -c 'mv "$1" "`echo $1 | tr "A-Z " "a-z_"`"' {} {} \;

This pattern-matching is not fully "bullet-proof", but it works for me, both under tcsh and bash.

Here is another one, but this time I want to convert the first letter of each 'word' to uppercase (so that 01-my_song.ogg becomes 01-My_Song.ogg. I use this e.g. to convert the filenames of encoded audio files:

for i in *.mp3; do mv "$i" "`echo $i | sed 's/\([-_]\)\(.\)/\1\u\2/g'`"; done

• A document of over 70 handy "one-line" sed commands is available at http://sed.sourceforge.net/sed1line.txt.

Prefix numbers

Using filenames that start with numbers is an easy way to keep them in the "right" sequence, e.g. for batch processing. As an example, to assemble a brochure you might want to name your pages:

1_intro.pdf
2_toc.pdf
3_topic-A.pdf
...   

... you get the idea. A problem appears when you exceed 9 pages since pages 10...19 will be sorted right after page 1 (since they all start with "1").

Fortunately, there are several options:

• Start numbering with (at least) two-digit numbers right from the beginning: 01_intro.pdf, 02_toc.pdf, ...
• Process files using the natural sort option of the ls command: pdftk `ls -v` cat output brochure.pdf
• Prefix 1-digit numbers with a zero: for i in [1-9]_*pdf; do mv -v $i 0${i}; done

Swap parts of filename

To swap parts of a filename, e.g. to rename 'foo-bar.201504.some.ext' to '201504.foo-bar.some.ext':

  rename 's/([a-z\-\_]+)\.([0-9]+)\.(.*)/$2.$1.$3/' file*

... and the other way round, i.e. to rename '201504.foo-bar.some.ext' to 'foo-bar.201504.some.ext':

  rename 's/([0-9]+)\.([a-z\-\_]+)\.(.*)/$2.$1.$3/' file*

The regex [a-z\-\_] matches all lowercase letters plus the hyphen and underscore characters.

Here is a more universal one, only relying on the dots as separators. This one will e.g. rename 'letter.en.template.lyx' to 'template.letter.en.lyx':

  rename  's/(.*)\.(.*)\.(.*)\.(.*)/$3.$1.$2.$4/' *template*lyx

Extracting image filenames from files

We often need to extract the exact filenames of image files from text files (e.g. when compiling notes). The following one-liner will extract all words containing jpg from the file foo.txt:

  grep '.jpg' foo.txt | grep -o '[^[:space:]]*jpg'

However, this may leave some additional chars in the filename, such as parentheses. We can clean these up with sed (sed 's/(//g;' for example). If we always use consistent filenames (such as the ISO 8601 notation YYYYMMDDThhmmss), we can use an even "tighter" regex of course. The following sed expression will remove any characters that are not alphanumeric or the dot sign:

  grep '.jpg' foo.txt | grep -o '[^[:space:]]*jpg' | sed 's/[^[:alnum:].]//g;' 

Even better: the following sed script variant will remove anything that is not (a) a digit, (b) the ISO 8601 T separator char or (c) the .jpg file extension:

  grep '.jpg' foo.txt | grep -o '[^[:space:]]*jpg' | sed 's/[^[:digit:]T.jpg]//g;'

As usual, the output can then be redirected to a file with > images.txt.

To copy exactly these files into the directory img/:

  xargs -a ../images.txt cp -vt img/

or feed them directly to my makealbum.sh script:

  xargs -a ../images.txt makealbum.sh

Zipping files

The usual archive format in the Linux world is the compressed tar archive, .tar.gz or .tgz. You can easily create "zip" files on linux. To deflate and compress all data files (say, *.raw) in a directory, run in bash:

   for i in *.raw; do zip -rmT9v ${i}.zip $i ; done

The option -r travels recursively, -m removes the original files after successful verification with -T, -9 is the highest compression level and -v is verbose mode. In contrast to WinZip, this runs nicely from the command line and can work on a sequence of files without efforts. Just make sure not to invert the position of the zip file and of the data-to-be-archived ;-)

You can also read the data-to-be-archived from a file, as long as this file contains one filename per line. Also, note that you do not always need to specify the extension .zip to the archive file, it will be added automagically:

   zip archive -@ < list.txt

The -@ option makes zip read from STDIN. This option can also be used to daisy-chain output from other commands such as find or grep. The following code takes the photos from the example above and packs them into a zip file named photos.zip:

   grep '.jpg' foo.txt | grep -o '[^[:space:]]*jpg' | sed 's/[^[:digit:]T.jpg]//g;' | zip photos -@ 

Un-zipping files from Windows

When you unzip "zip" files that were created on MS-Windows computers, filenames with special characters (Umlaute, accents etc.) may come out garbled. In this case, use another tool to unpack these files:

   ark -ab file_with_windows_encoding.zip

Option -a automatically creates subfolders (if the archive contains more than one folder), -b runs in batch mode. If you want to extract into a subdirectory:

   ark -ab -o targetdir file_with_windows_encoding.zip

Random sequences

For various projects that I have been working on, I needed to generate sets of random numbers where every number appears exactly once ... quite similar to drawing cards from a deck of cards.

The Perl script below is called as ./randomseq.pl min max. It will print all integers from min to max in random order.

Download the randomseq.pl script.

scp - Secure Copy

If you need to copy a file (or a directory) to a remote computer, scp is a secure replacement for rsh. The syntax is somewhat unusual but nevertheless straightforward:

• scp localfile remote: copies the file localfile to the home directory of the same user on the computer remote.
• scp localfile remote:text copies the file localfile to the directory text inside the home directory of the same user on the computer remote.
• scp -C -p -r -P 2222 localdir remoteuser@remote:/target/dir uses the account name remoteuser to access the computer remote and recursively (-r) copies the entire directory localdir to the directory /target/dir on the remote computer. Option -p preserves modification and access times, -C activates compression during the transfer. Option -P 2222 specifies that ssh is listening on port 2222 instead of the default, 22.

Identify all devices on a subnet

Using nmap to find all devices connected to a particular network:

nmap -sP 192.168.1.0/24

I'm using this frequently when I have to do remote maintenance on a system that is "inside" a remote network.

Identify that system

To find out what Linux version is installed on a particular system, simply look at the files /etc/*-release: cat /etc/*-release

Identify that server

To find out what software is driving a certain webserver, do a telnet to port 80. Enter HEAD / HTTP/1.0 and hit <enter> twice.

X login window

If you are using xdm as display manager, you may define a background image in /etc/X11/xdm/Xsetup ... e.g. with a line like background=/root/graphics/tux.xpm.gz.

The xdm configuration is defined in /etc/X11/xdm/xdm-config. Note that ~/.xsession contains the X server launch instructions if the login is done in X mode, while ~/.xinitrd contains the setup if the login is done in text mode and X is then launched manually with startx.

Two X Servers

Usually you start one X Server that is running on virtual console 7. What if you want to have two X Servers? No problem:

startx -- :0 &
startx -- :1 &

The -- is used by startx and xinit to separate an optional set of client parameters from the set of display/server options and parameters.

startx windowmaker -- :1 &

starts XWindows on virtual console 8 and runs windowmanager. Note that you should be logged in before doing so and that starting the same window manager twice from the same user account may confuse the configuration files ...

For more information, see the manpages to startx and xinit.

Sending fax

If you use HylaFAX, just type:

sendfax -n -D -s a4 -d 021xxxxxx /path/to/file.ps

... where -n means "no coversheet", -D mail notification to user upon delivery, -s specifies the paper size (here, a4) and -d is followed by the destination number.

HylaFAX sometimes hangs and in /var/spool/fax/status you can find a status message telling "waiting for modem to come ready". In such a case, with an outgoing fax in the queue, try the command faxmodem /dev/modem (indeed, this command should be in the script that launches the HylaFax server).

If you use a small system such as a portable computer and send faxes only occasionally, have a look at efax. This is a small but powerful package e.g. for use with a laptop, which requires only a few kilobytes of disk space and can be configured easily. Usage is very simple: to send a fax, just type

fax send 021xxxxxx /path/to/file.ps

If you need to receive a fax, use fax receive. You can even set this software up as a "fax printer" queue ... see the manpage for details.

Acessing MS-Windows resources: Samba Client

The key to using a Linux PC in a Microsoft Windows Network is the samba protocol. To gain access to the ressources of the NT network, you only need the samba-client package: The "full" samba-package contains also the server, but is only required if the Linux machine is to provide services towards the MS-Windows domain.

The configuration file /etc/samba/smb.conf must provide at least the following entries in the [global] section:

    workgroup = NAME_OF_THE_WINDOWS_DOMAIN
    encrypt passwords = Yes
    update encrypted = Yes

Now, to mount the NT/w2k "network drive" //server/directory, issue the following command as root:

mount -t smbfs -o username=XXX //server/directory /mnt/smb

... where XXX is your Windows username (btw, this option is not required if you are logged with the same name under Linux). You will be asked the corresponding password for your Windows logon; enter it and access should be granted. - Note that you can also give the password on the command line, however, this is unsecure: it could reveal your password to others that access the "history" function.

On recent Linux distributions, mount.smbfs is being replaced by mount.cifs; CIFS stands for Common Internet File System and is the successor to the SMB protocol. The syntax of the command stays the same, but you may need to use of the server's IP adresses instead of its name:

mount -t cifs //ip.of.server/directory /mnt/smb/

If you want a "permanent" mount in /etc/fstab:

//server/directory   /mnt/smb   cifs   guest,_netdev,uid=joe,gid=users   0 0

Any credentials issue is bypassed by the option guest and access is mapped to user 'joe'. The option _netdev is recommended for cifs mounts in fstab; it delays mounting until the network has been enabled.

If you want to use samba to access a directory on a Windows machine without using your "personal" WinNT domain account (e.g. to have a backup daemon watch a directory, without the need to enter the password every time you reboot that Windows box), proceed as follows (Note: this procedure is essentially identical for a computer running MS Windows 2000, but the entries are in slightly different locations.):

On the WinNT computer, create a new local user:

1. Start - Programs - Admininstrative Tools - UserManager
2. User - New
3. Enter a suitable Name (e.g. LinuxDaemon), description and password. Make sure the "Password never expires" field is checked; uncheck all other fields.
4. Under "Group", make this user a member of "BackupOperators" (not of "normal users").
5. Close the UserManager tool.

Still on the WinNT machine, adjust the share:

1. Open Windows Explorer (Win+E key).
2. Select the directory to be shared (e.g. c:\data\toarchive), right-click on it and select "Shared ...". Enter a suitable name and description.
3. Under "Permission", select "Add...". A new window opens.
4. From the drop-down list that appears on on top, select the name of the computer (not the name of the WinNT domain), e.g. \\PC12345.
5. Click on "Show Users", select "LinuxDaemon", then "Add". Under "Type of Access", select "Full Control".
6. 3x OK to exit.

On the Linux machine, we do not want the username/password to appear in clear text. Thus, create a file (e.g. /root/access.win) that holds the credentials and that is only readable to root. The file is very simple:

username = xxxxx
password = yyyyy

Now you can use the following mount command:

mount -t smbfs -o workgroup=PC12345,credentials=/root/access.win //PC12345/toarchive /mnt/smb

If you want this to be done automatically in /etc/fstab, use:

//PC12345/toarchive   /mnt/smb   smbfs   workgroup=PC12345,credentials=/root/access.win   0   0

Samba comes with lots of well written documentation; use it!

Providing resources to MS-Windows: Samba Server

If you work within a Windows NT domain, you will want to avoid that Linux "takes over" if the NT domain controller is rebooted. Thus, if you run a Samba server in the network, set the following parameters in the [global] section of the configuration file /etc/samba/smb.conf :

    workgroup = NAME_OF_THE_WINDOWS_DOMAIN
    encrypt passwords = Yes
    update encrypted = Yes

    domain master = no
    local master = no
    preferred master = no
    os level = 0
    domain logon = no

Restart SAMBA with /etc/rc.d/smb restart.

If you have a printer that is connected locally to your Linux box and you want to share it within a Windows NT/2000 network, the entries in your /etc/samba/smb.conf might look as above, plus a few more entries related to the printer:

[global]
    ...
    guest account = Nobody
    map to guest = Bad User

    printing = bsd
    printcap name = /etc/printcap

	# "load printers = yes" is only required if you want to browse
	# ALL printers on this system. In my case, I make only ONE printer
	# available (definition at the end of this file).
;       load printers = yes

[hp2200d]
    comment = HP2200d Duplex PS
    path = /tmp
    guest ok = yes
    writable = no
    printable = Yes
    printer = lp3
    browseable = yes

I had a lot of trouble getting my printers to work with MS Windows 2000. I finally found that the problem was (very probably) due to the fact that the spool directory var/spool/samba had not the correct access rights; since I changed it to /tmp everything works fine.

Using the configuration above in a Windows 2000 network, the Control Panel still shows the printer with "Access denied, unable to connect" ... but everything works fine, apparently even bidirectional printing and spooling.

Blank sheet after print job from MS-Windows. The printer queue "lp3" above is configured to print in RAW mode, so that all output from the MS-Windows printer driver is fed directly to the printer. However, I encountered the problem that I always got an extra blank page when a printjob was sent from MS-Windows, while printing directly from Linux always worked without that extra page. The solution was simply to remove the entry for the printer filter (e.g., apsfilter) from the corresponding entry in /etc/printcap ... a "raw" queue does not need a filter anyway ;-)

On this site

• GPS Software and Linux.
• My installation notes on Debian Buster and outdated notes on Fedora 14.
• Linux on one of my former laptops, a Fujitsu LifeBook B142.
• Some hints about scanning, image conversion, making picture albums etc. under Linux.
• Using a Linux PC as X-client for high-end workstations.
• s7150, a software to access and control the Solartron 7150 Digital Multimeter using GPIB under Linux.
• hp6633, a software to access and control the the Hewlett-Packard 6632, 6633, 6634 Power Supplies using GPIB under Linux.