< index >      
 Hands-on Guide to the Debian GNU Operating System

Hands-on Guide to the Debian GNU Operating System

Hands-on Guide to the Debian GNU Operating System

Davor Ocelic

Revision History
Revision 2.430 March 2002Revised by: docelic
This is the initial release.
Revision 2.502 May 2002Revised by: docelic
Spelling & grammar corrections, Spanish translation (all by Walter Echarri)
Revision 2.603 June 2002Revised by: docelic
Content improvements, Walter's grammar corrections.
Revision 2.724 July 2002Revised by: docelic
The lyx->sgml variant just didn't work, output documents were so bad-looking. I've re-written it manually in docbook sgml, with newbiedoc stylesheets.
Revision 2.812 September 2002Revised by: docelic
Thanks to Gürkan Sengün (gurkan/@/linuks.mine.nu) for various good tips, suggestions and links (and hosting the guide!)
Revision 2.915 September 2002Revised by: docelic
Improvements; links to external resources.
Revision 3.016 September 2002Revised by: docelic
Improvements; looks fine now.
Revision 3.16 January 2003Revised by: docelic
Spelling and style corrections by Walter Echarri.
Revision 3.29 Feb 2003Revised by: docelic
Added the 'Debian GNU kernels' chapter.
Revision 3.38 Mar 2003Revised by: docelic
Refactoring, more small updates
Revision 3.418 Mar 2003Revised by: docelic
Added details, corrected sgml source bits

Table of Contents
1. Table of Contents
2. Introduction
2.1. Official "Hands-on Guide" download sites
2.2. Acknowledgements
3. Conventions
4. Basic theory
4.1. Unix
4.2. Free Software, GNU and Linux
4.3. GNU kernel - The Hurd
4.4. The Debian GNU system, its design goals and basic ingredients
5. Basic system management
5.1. Package management
5.2. Getting familiar with system messages
5.3. The X Window System, basic principles and Debian setup
5.4. Virtual consoles
5.5. Shutting down the system
5.6. General notes for hardware support
5.7. Enabling the mouse in text consoles
5.8. Hard disk throughput
5.9. Monitoring non-free software on your machine
5.10. Firewalls
5.11. Setting up IP Masquerading/NAT
5.12. System login procedure, the shell startup and config files
5.13. Regular user accounts
5.14. Switching to root account without a password
5.15. Account login regulation
5.16. Tcp wrappers
5.17. Manually unpacking .deb files
5.18. Checking the MD5 sums of installed Debian packages
5.19. Shared sessions
5.20. Runlevels and system services
5.21. The Debian 'alternatives' system
5.22. Periodically checking for the available disk space
5.23. Creating and extracting file archives
5.24. Copying, mirroring and re-downloading Debian packages
5.25. Package recompilation
5.26. Linking to your local Internet Service Provider
5.27. The package popularity contest
5.28. Accessing data on MS Windows partitions
6. Linux processes
6.1. Introduction
6.2. Basic process-related commands
6.3. Two distinct types of executable files
6.4. How to start a process
6.5. How to terminate a process
6.6. How to put a process to sleep?
6.7. Process priorities
6.8. Processes and their input/output functions
6.9. Leaving processes running while you're away
7. Using Debian GNU
7.1. Common keystrokes
7.2. Terminal settings
7.3. Learning and using the vim editor
7.4. The readline key and function bindings
7.5. User configuration files
7.6. Command aliases
7.7. Advanced command line features
7.8. Customizing the X session
7.9. Reporting bugs
8. Debian GNU kernels
8.1. Basic kernel information
8.2. Kernel recompilation
8.3. Kernel image installation
8.4. System bootloader
9. Try to do it yourself first
9.1. A generic tasklist
9.2. getting help on IRC
9.3. How not to ask questions on IRC
9.4. Frequently used terms

2. Introduction

The Guide is available under the terms of the GNU GPL license, and you should probably read it after you successfully install the Debian GNU system on your computer (with or without the help of the Debian installation guide).

This is a step-by-step document with many examples, which should relatively quickly answer most of your questions and help you build the correct mindset to solve further problems on your own; I am known for repeating that the idea and logic count, not the exact implementation or usage details (I am all for the "give man a fish and he can eat today, teach a man to fish and can eat forever" principle here).

I tried to make it a balanced mix bewteen the administrator's and the user's guide; it is probably too broad for those who belong to either of the two extreme categories. The approach I used should fit home users best - people who do have a Debian installation and a root access at hand, and want to learn and experiment.

We will properly define basic terms, explain the system design goals, cover the most important end-user issues and show many command line examples. Since this is a Debian guide, we will not hesitate to use Debian-specific features and commands, but note that most of it (ideologically, at least) applies to other Linux or Unix systems as well. Finally, by saying this is a beginner's guide, we definitely don't restrict ourselves to system basics, I believe the guide is hiding many beautiful details that even experienced users might find useful or amusing.

Please note that all the fine information presented here can also be found in respective packages' documentation and is more detailed and comprehensive there. Therefore, it is implicitly suggested to read official software and system documentation in combination with this guide (the dpkg(8) and apt(8) manual pages are perfect to show there's much more of it than we mention here). Generally, www.tldp.org (former linuxdoc), www.debian.org and www.debian.org/doc, /usr/{doc, share/doc, local/share/doc} directories, and the man and info pages on your system are the right information sources.

After you finish reading this guide, you'll probably want to read other on-topic manuals available from the Debian documentation directory.


2.2. Acknowledgements

  • Walter Echarri <wecharri/@/infovia.com.ar>: the Spanish translation, extensive proof-reading, numerous grammar/style corrections, and motivation. Thanks Wally ;)

  • Adam Garside <asg/@/gimp.shacknet.nu>: proof-reading and grammar/style corrections.

  • Gürkan Sengün (gurkan/@/linuks.mine.nu) for various good tips, suggestions and links (and hosting the guide!).

  • Thanks to all who sent their comments, suggestions or updates.


3. Conventions

  • All system commands are emphasized (free, top, ps) and optionally given in single quotes ('rm'). In case we are referring to the program manual pages, the names also include the section of the manual, as you see from the dpkg(8) and apt(8) examples.

  • All package, file and directory names are emphasized and start with / (slash) or ~ (tilde): /etc/init.d/, ~/.bash_profile

  • Symbols like [PID] indicate you should replace [PID] with the real value of "PID"; for example, kill -9 [PID]

  • Inside chapters, text between "[" and "]" angle brackets provides short explanations, [like the example here]

  • In code and command-line examples, all user input is prefixed with $ (for user commands) or # (for commands which require root permissions). Program output is edited for brevity and has no prefix. While this might prevent you from directly copy-pasting the instructions to your shell, it is always a good thing to use your brain and ponder while re-typing.

  • Unix, GNU, Debian and Linux are words that can sometimes, depending on the context, be used interchangeably. Throughout the guide, I tried to be consistent in always using the word with the broadest scope for which a given note applies. For instance, we would talk about the Unix command line, GNU tools, Debian-way of doing things and Linux process management.


4. Basic theory

Let's use this chapter to identify some common misinterpretations and properly define the terms we will use everywhere in this document. We'll start with general terms, such as Unix, GNU or Free Software, and then say something about the Debian Project itself. A little glossary of the various terms that you'll be encountering is provided in the appendix.

Let's start.


4.1. Unix

In earlier versions of this document, I used to say that Unix was a common name for a group of superior operating systems which shared most of the key design ideas. While there was nothing wrong with that statement, I went to search the Internet for some more formal explanations:

Short introduction on the UGU site says:

Unix - /yoo'niks/ Plural "Unices". An interactive time-sharing operating system invented in 1969 by Ken Thompson after Bell Labs left the Multics project, originally so he could play games on his scavenged PDP-7. Dennis Ritchie, the inventor of C, is considered a co-author of the system.

Similar and more detailed description from searchSolaris:

Unix is an operating system that originated at Bell Labs in 1969 as an interactive time-sharing system. Ken Thompson and Dennis Ritchie are considered the inventors of Unix. The name (pronounced YEW-nihks) was a pun based on an earlier system, Multics. In 1974, Unix became the first operating system written in the C language. Unix has evolved as a kind of large freeware product, with many extensions and new ideas provided in a variety of versions of Unix by different companies, universities, and individuals.

Partly because it was not a proprietary operating system owned by any one of the leading computer companies and partly because it is written in a standard language and embraced many popular ideas, Unix became the first open or standard operating system that could be improved or enhanced by anyone. A composite of the C language and shell (user command) interfaces from different versions of Unix were standardized under the auspices of the IEEE as the Portable Operating System Interface (POSIX ). In turn, the POSIX interfaces were specified in the X/Open Programming Guide 4.2 (also known as the "Single Unix Specification" and "Unix 95"). Version 2 of the Single Unix Specification is also known as Unix 98. The "official" trademarked Unix is now owned by the The Open Group, an industry standards organization, which certifies and brands Unix implementations.

 


4.2. Free Software, GNU and Linux

Some time later, Richard Stallman, a MIT hacker, started an initiative to create a completely free operating system (free as in freedom). Among other things, his decision was based on frustrations and problems he saw in non-disclosure agreements. They once prevented his colleague from giving him the source code for a laser printer driver (Stallman wanted to include automatic paper-jam notification features).

Highly motivated to do The Right Thing (tm), he later quit the job at MIT (so they couldn't possibly claim copyright on his work) and, in 1984, started the GNU ("Gnu's Not Unix") project, whose goal was to protect freedom and supply users with full-featured free software packages for their computers. GNU is a wonderful philosophy that could surely affect non computer-related areas as well.

You can see the original Stallman's announcement from Sep 27, 1983 / 10:35:59 PST in the excellent Google Groups archive!

GNU developers have re-written all the necessary Unix system tools and utilities, released them as Free Software (under the GNU GPL licence), and they only needed a kernel to accomplish the initial goal.

Independently, in 1991, Linus Torvalds (from the Helsinki University) announced his first public release of the kernel he was working on - Linux. He was a student back then, and wanted to create a cheap alternative to high-priced Unix systems, which would run on PC (i386) compatible machines. Combining the Linux kernel and the GNU tools, the free GNU/Linux system became a reality. Linus wrote the kernel from scratch ("from zero") and it was one of the first free Unix-like variants which, supported by the great GNU community and their software, got the Free Software movement really going (from the general-public perspective, not technically, of course).

All the way back in 1994, Peter van der Linden wrote the following in his excellent book, titled “Expert C Programming; Deep C Secrets” (ISBN 0-13-177429-8):

The Free Software Foundation is a unique organization founded by ace MIT hacker Richard Stallman. By the way, we use “hacker” in the old benevolent sense of “gifted programmer”; the term has been debased by the media, so outsiders use it to mean “evil genius”. Like the adjective bad, “hacker” how has two opposing meanings, and you have to figure it our from the context.

Stallman's Free Software foundation was founded on the philosophy that software should be free and freely available to all. FSF's charter is “to eliminate restrictions on copying, redistribution, understanding and modification of computer programs” and their ambition is to create a public-domain implementation of Unix called GNU (it stands for “GNU's Not Unix”. Yes, really).

Many computer science graduate students and others agree with the GNU philosophy, and have worked on software products that FSF packages and distributes for free. This pool of skilled labor donating their talent has resulted in some good software. One of the FSF's best products is the GNU C compiler family. gcc is a robust, agressive optimizing compiler, available for many hardware platforms and sometimes better than the manufacturer's compiler.

However, there were other efforts, such as those by the BSD people who still had problems with the licensing issues and copyrights, but they have rewritten all the parts in question and released free BSD variants: FreeBSD, NetBSD and OpenBSD.

NotePlease Note:
 

Linux (and other free operating systems today) have picked up the best from the Unix world and additionally, they have many end-user advantages over the orthodox Unix machines (primarily in aspects of "user friendlyness" and GUI environments).


4.2.1. Open Source

Open Source is a somewhat newer term which was generally accepted to help promote Free Software in commercial environments. It relies only on practical benefits of open source code (quality, reliability, cost of maintenance) and has no greater philosophy behind it. More information can be found at the Open Source Initiative (OSI) website.

It is therefore important to know the disctinction between the two.

ImportantImportant!
 

Read more about the Free Software, Open Source and the correct interpretation of the word free on the Debian's What Does Free Mean? page.


4.3. GNU kernel - The Hurd

It is interesting to mention that Linux is a monolithic kernel and shares many ideas with its Unix counterparts. However, the GNU people have a different vision of how kernels should look like and they are working on The Hurd microkernel. Debian GNU/Hurd port is in progress, and you can see the current status or download the software from the Debian GNU/Hurd port page.

Monolithic and microkernels are fundamentally different, and there's been much of debate if microkernels would ever prove useful in real-life application. Linus Torvalds, for example, is constantly bashing microkernel operating systems ("just say NO to drugs, and maybe you won't end up like The Hurd people"). Alan Cox, the maintainer of the production tree of the Linux kernel, who has more sympathies for The Hurd, once said that The Hurd was more about Richard Stallman's idea about how a system should work to promote community than about high perfomance OS design.

Technically, The Hurd and microkernels in general do offer many advantages over the traditional Unix kernels; those interested in getting more information should see hurd-paper.html and hurd-talk.html (for The Hurd), or the QNX website (for the proprietary, mature, microkernel-based Unix).


4.4. The Debian GNU system, its design goals and basic ingredients

Let's quote something from the official About Debian page:

Debian was begun in August 1993 by Ian Murdock, as a new distribution which would be made openly, in the spirit of Linux and GNU. Debian was meant to be carefully and conscientiously put together, and to be maintained and supported with similar care. It started as a small, tightly-knit group of Free Software hackers, and gradually grew to become a large, well-organized community of developers and users.

Since many people have asked, Debian is pronounced 'deb ee n'. It comes from the names of the creator of Debian, Ian Murdock, and his wife, Debra.

Debian is produced by nearly one thousand developers spread around the world who volunteer in their spare time. Few of the developers have actually met in person. Communication is done primarily through e-mail (mailing lists at lists.debian.org) and IRC (#debian channel at irc.debian.org).

The Debian Project is an association of individuals who have made common cause to create a free operating system. This operating system that we have created is called Debian GNU/Linux, or simply Debian for short.

An operating system is the set of basic programs and utilities that make your computer run. At the core of an operating system is the kernel. The kernel is the most fundamental program on the computer and does all the basic housekeeping and lets you start other programs.

Debian systems currently use the Linux kernel. Linux is a completely free piece of software started by Linus Torvalds and supported by thousands of programmers worldwide.

However, work is in progress to provide Debian for other kernels, primarily for the Hurd. The Hurd is a collection of servers that run on top of a microkernel (such as Mach) to implement different features. The Hurd is free software produced by the GNU project.

A large part of the basic tools that fill out the operating system come from the GNU project; hence the names: GNU/Linux and GNU/Hurd. These tools are also free.

Of course, the thing that people want is application software: programs to help them get what they want to do done, from editing documents to running a business to playing games to writing more software. Debian comes with over 8710 packages (precompiled software that is bundled up in a nice format for easy installation on your machine) -- all of it free.

It's a bit like a tower. At the base is the kernel. On top of that are all the basic tools. Next is all the software that you run on the computer. At the top of the tower is Debian -- carefully organizing and fitting everything so it all works together.

You may be wondering: why would people spend hours of their own time to write software, carefully package it, and then give it all away? The answers are as varied as the people who contribute. Some people like to help others. Many write programs to learn more about computers. More and more people are looking for ways to avoid the inflated price of software. A growing crowd contribute as a thank you for all the great free software they've received from others. Many in academia create free software to help get the results of their research into wider use. Businesses help maintain free software so they can have a say in how it develops -- there's no quicker way to get a new feature than to implement it yourself! Of course, a lot of us just find it great fun.

Debian is so committed to free software that we thought it would be useful if that commitment was formalized in a written document. Thus, our Social Contract was born.


5. Basic system management

As we've just covered a bit of the theory, we'll now move on to the basic system administration issues.

We will start with basic topics (with software installation, for example), then cover the hardware configuration principles (including the graphical X Window System), suggest generally good things to do with fresh Debian installations and provide help and examples for common software packages.


5.1. Package management

The first thing we will take a closer look at is the Debian package management system. We'll take a tour of dpkg(8), apt(8) and other package management related tools.

What we're referring to when we say package management is the set of tools we use for browsing, installation, configuration and removal of software packages.

NotePlease Note:
 

All the package management backends and frontends (dpkg, apt-get, synaptic, aptitute, dselect, ...) use the same package database. This means that the changes made by one are seen by other tools as well, and you can therefore combine them all.

If you're not specifically interested in all the details and command line switches at the moment (that is, if this is just your easy late night reading ;-), then just briefly remember the command names and their purpose, and skip down to the Getting familiar with system messages subchapter.


5.1.1. dpkg

dpkg is a medium-level package manager for Debian. Unless you run into problems with apt-get, you will generally not have to use it directly.

Most notably, dpkg does not have the automatic package retrieval methods, and does not resolve dependencies on its own.

-i vim_6.0.093-1.deb

install package vim, version 6.0.093, Debian revision 1. Two things to note here:

  1. You need to know where the .deb file is located, and provide the path to it.

  2. dpkg does not check for dependencies so vim could be unpacked, but its configuration would be delayed until you install all the required packages (which is a boring and generally stupid job to do manually; see apt-get below).

-r vim

Remove package vim if there are no installed programs that depend on vim; leave configuration files on the system.

--purge vim

Remove package vim and all its configuration files.

--configure vim

Configure package vim.

--configure --pending

Configure all pending packages.

--get-selections

Retrieve current package states from the dpkg database.

--set-selections

Set package states. Accepts output generated from the option above. Of course, it can also be used on a per-package basis; echo “vim hold” | dpkg --set-selections would set package vim 'on hold'. Once you load this list with --get-selections, use apt-get dselect-upgrade to actually make the packages [de]install on the system.

--force-depends

Option which could be used everywhere with dpkg, but it almost always leads to dpkg database corruption (specifically, version mismatches) and total dependency chaos. If you later plan to use apt-get, never use this option as it instantly breaks apt (you can, however, use apt-get -f install and apt will do its best to clean up the mess).

-l

produce package list (keep an eye on the desired and status (first two) columns)

-S /path/to/file

find out which package does file belong to

-L vim

show all files installed by package vim

--status vim

Status information for the package vim, similar to apt-cache show


5.1.2. dpkg-reconfigure

dpkg-reconfigure is a tool you use to reconfigure debconf-enabled packages (those which use debconf to ask questions and get answers about the local configuration).

dpkg-reconfigure gpm

reconfigure package gpm. Only applicable if the package (gpm in this case) relies on debconf.

dpkg-reconfigure debconf

reconfigure debconf itself. You can choose between few types of interactive or non-interactive package configuration modes. Non-interactive mode is very useful if you are performing mass or automated installations.

TipTip
 

Sometimes (due to a bug in a specific package's debconf interface), you won't be able to successfuly configure the package; this is very likely to happen from time to time if you use the Debian unstable tree. Common examples would be the 'Accept' buttons which don't actually accept the input or text fields which are always considered empty. A possible hack solution for this kind of problem is to reconfigure debconf to non-interactive, then configure the problematic package and finally reconfigure back to some sort of interactive mode.

TipTip
 

You will most probably be using this command to reconfigure the X Window System every now and then, so just remember this command, which is the elegant Debian-specific way to deal with the configuration: dpkg-reconfigure xserver-xfree86


5.1.3. apt

The apt package provides a few command-line tools you will need to successfully use apt-get(8), the tool for high-level package management.


5.1.3.1. apt-setup

apt-setup (beware, from the base-config package) opens up a ncurses-based apt configuration tool. Basically, it asks a series of questions and then updates the package files (you can do the same manually by editing /etc/apt/sources.list and running apt-get update).

Also check the netselect package, which should select the fastest mirror servers for you automatically.

It can sometimes come handy to do telnet linuks.mine.nu | tail -n 5 > /etc/apt/sources.list to retrieve the apt sources for the Debian unstable branch.

For more exotic apt sources, check www.apt-get.org.


5.1.3.2. apt-cdrom

If you have the packages on your cdroms, you will use the apt-cdrom utility to index them.

# apt-cdrom add


5.1.3.3. apt-get

apt-get is an apt package handling utility. It is probably the most convenient way to install or remove packages, as it automatically calculates dependencies and adjusts package lists.

While dpkg allows you to install any .deb file (provided you have the appropriate .deb file saved locally), apt-get does not. It uses the /etc/apt/sources.list file as its list of 'package sources'; it parses them and creates a big list of all available packages. So whatever you do, you're restricted to packages known to apt. This is both powerful and elegant way to deal with package management, and some of the complicated tasks (such as the package or whole distribution release upgrades) become so easy with Debian GNU that you will hardly believe it!

Other, rpm-based distributions are trying to catch up with apt, either by reimplementing the logic in their own programs or porting apt to their systems, but of course, Debian always knows better so stay with the winning team.

update

Retrieve and update the package lists. Call this every time you change the /etc/apt/sources.list file, or on a daily basis if you use the remote apt sources (those that might update their contents).

install vim

Install package vim. apt uses its internal database to find out where is the package located (it could be on some of your CDs, the Internet or local apt mirror).

apt-get install vim=6.0.093-1

The same as above but installs version 6.0.093-1 specifically (use apt-cache to see available versions for a package).

--reinstall install vim

Self-explanatory now.

remove vim

Remove package vim (and other packages which strictly depend on it).

--purge remove vim

Completely remove package vim and other packages which strictly depend on it. This also removes config files, which normally stay on the system.

upgrade

Upgrade packages on the system. Whether there are any candidates for upgrade depends on the apt mirror and your local database (for example, if you set a package on Hold, it will be skipped). You can use the -s switch to just check out which packages would actually get upgraded.

Some time ago, I read about RedHat's tool which allowed Internet updates, but required you to first send a list of all packages you have on your system (so that RedHat server could compare it to the database and then report new packages available). Apt doesn't work that way - it anonymously grabs whole lists and then locally decides which packages you are interested in. You would think that's the only reasonable way to do an update, but then you see people from RedHat ...

TipTip
 

To upgrade specific packages only, either re-run apt-get install [package names], or run debfoster -u [package].

-f install

fix Debian installation; perform necessary steps to get internal database back in order. Handy when you previously mess it up with dpkg --force-depends (although it usually means package downgrades or mass deinstalls).


5.1.3.4. apt-cache

apt-cache can be used to query the dpkg package database.

show vim

show internal information on package vim (version, sizes, dependencies, conflicts, suggests, description ...)

search vim

search the database for package names or descriptions that contain vim.

--names-only

only search package's Name field (otherwise it looks in all fields).


5.1.3.5. apt-rdepends

apt-rdepends performs recursive dependency listings similar to apt-cache.

It searches through the APT cache to find package dependencies, and it knows how to emulate the result of calling apt-cache with both depends and dotty options. By default, it shows a complete dependencies listing.


5.1.3.6. Graphically representing package dependencies

apt-cache dotty takes a list of packages on the command line and generates output suitable for use by dotty from the GraphVis package. The result will be a set of nodes and edges representing the relationships between the packages. By default the given packages will trace out all dependent packages which can produce a very large graph. This can be turned off by setting the APT::Cache::GivenOnly option (man apt_preferences).

The resulting nodes will have several shapes, normal packages are boxes, pure provides are triangles, mixed provides are diamonds, hexagons are missing packages. Orange boxes mean recursion was stopped [leaf packages], blue lines are pre-depends, green lines are conflicts.

# apt-cache dotty vim | dot -Tps 
>

CautionCaution
 

dotty cannot graph larger sets of packages.


5.1.4. grep-dctrl

The grep-dctrl utility greps Debian control files.

The grep-dctrl program can answer such questions as

  • What is the Debian package foo?

  • Which version of the Debian package bar is now current?

  • Which Debian packages does John Doe maintain?

  • Which Debian packages are somehow related to the Scheme programming language?

  • and with some help, Who maintains the essential packages of a Debian system?

It is a specialised grep program that is meant for processing any file which has the general format of a Debian package control file, as described in the Debian Packaging Manual. These include the dpkg available file, the dpkg status file, and the Packages files on a distribution medium (such as a Debian CD-ROM or an FTP site carrying Debian).

For instance, too see all the packages for a maintainer, do:

$ grep-dctrl --show Package --field Maintainer 'Maintainer Name' /var/lib/apt/lists/*

For a lot more usage examples, see the grep-dctrl(1) man page.


5.1.5. debfoster and deborphan

Tools to weed out unnecessary Debian packages. Their use is trivial.

For example, to remove all unnecessary packages, you could do:

# apt-get --purge remove `deborphan`


5.1.6. dpkg-repack

dpkg-repack package provides us with a tool to bundle installed packages back into the .deb format. If any changes have been made to the package while it was unpacked (ie, files in /etc modified), the new package would inherit the changes.

This utility can make it easy to copy packages from one computer to another, or to recreate packages that are installed on your system, but no longer available elsewhere.

# dpkg-repack vim


5.1.7. dpkg-divert

dpkg-divert overrides a package's version of a file. File diversions are a way of forcing dpkg not to install a file into its location, but to a different location. Diversions can be used through the Debian package scripts to move a file away when it causes a conflict. System administrators can also use it to override some package's configuration file, or whenever some files (which aren't marked as 'conffiles') need to be preserved by dpkg, when installing a newer version of a package which contains those files.

I used it in our ttysnoop+ssh setup (see below):

# dpkg-divert --divert /bin/login.real --add /bin/login


5.1.8. dpkg-statoverride

stat overrides are the way to tell dpkg to use a different owner or mode for a file when a package is installed. This can be used to force programs that are normaly setuid to be installed without a setuid flag, or only executable by a certain group.

See the dpkg-statoverride(8) man page for details.


5.2. Getting familiar with system messages

It is very important to learn how does the system communicate with its users (or administrators). One can always find the exact source of the problem and take appropriate actions (simple, proven-to-be-useful tasks help in almost any situation), so this is why the chapter had priority in the final document layout.


5.2.1. Boot messages

During the boot, the system kernel prints out a lot of interesting information (unless the quiet option was passed to it). The copy of the messages is saved in the /var/log/dmesg file (which does not grow with time).

The dmesg command, however, will show you the last 4 KB of recent kernel messages.


5.2.2. System logging daemon

Unix machines have a standardized way for programs, applications and daemons to send messages to the global system logger (syslog). There are many syslog implementations available; with Debian, you can choose betweeen the default traditional BSD sysklogd, syslog-ng and metalog.

Each message has an indication of the facility (message source) and severity (importance level). The date, time, host and process information is automatically generated by syslog, and should not be a part of the message itself. The syslog daemon distributes messages to files, pipes, remote destinations or users, using the schema specified in the /etc/syslog.conf file (for the traditional BSD sysklogd).

All the logs from a vanilla ("out of the box") Debian system are written to files in the /var/log/ directory.

NotePlease Note:
 

Nothing prevents you from specifying multiple destinations for the same message.

To collect all system messages (for strictly educational purposes :) in a single file, add a line like this in /etc/syslog.conf:

*.* TAB /var/log/allmessages

[TAB is there to warn you that you really have to press the TAB key, spaces don't do it right]. Then create an empty /var/log/allmessages file (choose your favorite, both variants here do the same):

# touch /var/log/allmessages
>

And just reload the sysklogd daemon configuration:

# /etc/init.d/sysklogd reload

Now go to some idle virtual console, and type (see the tail(1) man page):

$ tail -f /var/log/allmessages

Do something to your system (for example, logout, login or use 'su' on another console, and watch messages appear!). This is an excellent way to learn more about the system and how it works. Also, you can detect any anomalies and error reports that would otherwise go unnoticed.

If you are writing shell scripts, or modifying your ~/.bash_profile, you can use the 'logger' command to log your messages via syslog.

NotePlease Note:
 

I prefer echoing all messages to /dev/tty12 which makes them easy to check just by switching to the virtual console 12 (Alt+F12). If you are using the X graphical interface most of the time, check out the root-tail package which monitors log files and prints messages to your root window (the background). You can also log messages to pipes and then read them with a GUI application, or use some of the standard tools which simply look at the files in /var/log.


5.2.3. Logging ppp messages

It is nice to have all ppp logs go to /var/log/ppp.log; the 'plog' command will then work as expected. The following will add a line to /etc/syslog.conf and restart the syslog daemon (we used the traditional BSD sysklogd in the example):

# echo -e "local2.*\t/var/log/ppp.log" >> /etc/syslog.conf
>


5.3. The X Window System, basic principles and Debian setup

5.3.1. The XFree86 Project, an open-source X Window System implementation

From www.XFree86.org:

The XFree86 Project, Inc is the organization which produces XFree86, a freely redistributable open-source implementation of the X Window System which runs on Unix(R) and Unix-like operating systems such as Linux, all of the BSD variants, Sun Solaris x86, Mac OS X (via Darwin), as well as other platforms like OS/2 and Cygwin.

XFree86, the product, provides a client/server interface between display hardware (the mouse, keyboard, and video displays) and the desktop environment while also providing both the windowing infrastructure and a standardized application interface (API). XFree86 is platform-independent, network-transparent and extensible.

With XFree86 a user cannot only choose the desktop environment they prefer, but because we are an open-source project, users can also modify and update their systems as they see best. As XFree86 has always been an unabashed supporter of freedom of the user desktop, so we encourage users to customise and personalise their desktops with the application of their choice, whether it be KDE, GNOME, Enlightenment, Blackbox, AfterStep, fvwm or twm.

Our goal at XFree86 is to have X run on every platform available, including those we do not currently support, as the best windowing system available on that platform.


5.3.3. XFree86 Installation

Debian potato (2.2) is shipped with XFree86 version 3.6.6, while Debian woody (3.0) has XFree86 version 4. See current status at the X Strike Force homepage.

X4 brings a lot of improvements and is now standard in Debian. There's not much difference from administration perspective, but notes will be put where appropriate.

You can install basic X support, the icewm window manager and the wdm display manager with:

# apt-get install xserver-xfree86 xbase-clients xfonts-base icewm icewm-themes wdm 	

NotePlease Note:
 

If you're using X3, do apt-cache search xserver- and install the appropriate one instead of xserver-xfree86.

wdm is a better-looking equivalent to xdm, the X Display Manager (it opens up graphical login prompts). It is nice to have it, especially if you want to install more window managers, and then select which one to use from the wdm's menu. If you are using Gnome or KDE consider using their native gdm or kdm programs.


5.3.4. XFree86 Server Configuration

When you install the packages, the configuration process will start automatically. If you don't get it right the first time, you can always re-run configuration with:

# dpkg-reconfigure xserver-common
>

The interface is very clean and should help you create working XFree configs in no time. In case of problems, inspect the config file (/etc/X11/XF86Config-4 or /etc/X11/XF86Config) manually to make sure you have the right Driver option, and that UseFBDev option is set to false (these are the most common errors).

TipTip
 

If you see no UseFBDev option in the config file, you need to manually add it and set it to false. The proper location to do it is the Section "Device" part of the config file, and it needs to look more or look like this:

Section "Device"
    Driver      "ati"
    Option      "UseFBDev"   "false"
    BusID       "PCI:1:0:0"
EndSection

5.3.5. Tuning the resolution in X

When you start X, it picks the default color depth, loads in the list of available resolutions for the given depth, and displays the highest one. You can then cycle over other pre-defined resolutions with Ctrl+Alt+'+' and '-'.

All this is set up in /etc/X11/XF86Config-4. Here's an excerpt from the configuration for 16bit colors with default resolution 1024x768:

...
Section "Screen"
 ...
 DefaultDepth 16
 SubSection "Display"
   Depth 1
   Modes "1152x864" "1024x768" "800x600" "640x480"
 EndSubSection
 ...
 SubSection "Display"
   Depth 16
   Modes "1024x768" "800x600" "640x480"
 EndSubSection
 ...
EndSection
...

To explicitly start X with 16bit colors (if there's no DefaultDepth option or it is different), use:

startx -- -bpp 16

If you want to further experiment with refresh rates and resolutions, either manually tune VertRefresh, HorizSync and Modeline (in X3 only) definitions in X config file, or see OpenBSD's X tuning guidelines.


5.3.6. Device autodetection

To take advantage of some kind of device autodetection, see the following three programs:

  • read-edid, hardware information-gathering tool for VESA PnP monitors

  • mdetect, mouse device autodetection tool

  • discover, hardware identification system


5.3.7. The client-server model

Since X is a client-server based model (as are most other things in Unix), it means you have a whole new domain of features at your disposal. We'll discuss them now.

  • Typical local-user/local-machine session

    When you start X (with startx, xinit or X), it opens the first free virtual console (that is console 7 in most Linux distributions), and starts X server on it (X server uses the DISPLAY environment variable to detect the target display, and in this case it is “localhost:0”; just “:0” or undefined DISPLAY variable have the same effect). The X server then starts the window manager of your preference and the desktop screen shows up. All the files needed are found on your local disk. You can switch back to your console screens with Ctrl+Alt+F1, F2 etc... To get back to your X display, use Alt+F7. To close your X session, either find some form of a Logout button in your window manager, or simply use Ctrl+Alt+Backspace. Just as you can have more virtual text consoles, you can have more completely separate X displays on a single display device (of course, even under different usernames): To see it in practice, start X (with startx& command), then switch back to text console with Ctrl+Alt+F1, and run 'startx -- :1&'. Bravo! You have two X sessions running now! Switch between them with Ctrl+Alt+F7 and Ctrl+Alt+F8.

  • Remote displays on your machine

    Let's say you have two machines, Monarch and Denali. You are sitting at Denali, and would like to start some X program on Monarch, but have the display locally on Denali's monitor (Note that this isn't a common file sharing: in our case, the program is really executed on Monarch, only the display is sent to Denali). We will use one very convenient approach (there are other ways, of course) - we will use slogin program (an alias for ssh actually) to log in to monarch. The slogin command will set up the .Xauthority magic cookie file and the DISPLAY variable automatically, so all we need to do is to start our application. Try xeyes. Here's an example for convenience:

denali:~$ slogin monarch
Enter password: xxxxx
monarch:~$ echo $DISPLAY
denali:0
monarch:~$ xeyes&

NotePlease Note:
 

Note however that you must have the following options enabled for the example above to work: X11Forwarding in /etc/ssh/sshd_config and ForwardX11 in /etc/ssh/ssh_config. To restart the ssh daemon, use /etc/init.d/ssh restart


5.3.8. The Direct Rendering Infrastructure (DRI)

Here's a little introduction from the Documentation/Configure.help file (the kernel-doc-* packages):

AGP (the Accelerated Graphics Port) is a bus system mainly used to connect graphic cards to the rest of the system. If you have an AGP system, it will be possible to use the AGP features of your 3D rendering video card. Note that this is the only way to have XFree4/GLX use write-combining with MTRR support on the AGP bus. Without it, OpenGL direct rendering will be a lot slower but still faster than PIO.

Kernel-level support for the Direct Rendering Infrastructure (DRI) was introduced in XFree86 4.0 (which you do have, if you use Debian Woody (3.0) or newer releases).


5.3.9. XFree86 Notes

X3 has fewer drivers and you must install specific drivers for specific groups of graphic cards (for example, xserver-rage128, mach32, mach64, i128, 3dlabs, agx, 8514, s3v etc...). In X4, we solve this by only installing xserver-xfree86, which is modular and loads the appropriate modules at runtime. Also, the config file is /etc/X11/XF86Config-4 for X4, and just /etc/X11/XF86Config for old X3.

Generally, only use X3 on old machines where you want to save some memory.


5.3.10. Troubleshooting

  • Check the /var/log/XFree86.log and ~/.xsession-errors files for hints.

  • Edit /etc/X11/XF86Config-4 and search for the line Option "UseFBDev" "true" and turn 'true' to 'false'.

  • If it still doesn't work, edit Driver= config parameter.

  • After you make sure the driver option is ok, but it still doesn't work, try tweaking HorizSync and VertRefresh values. Try with this: HorizSync 30-80 VertRefresh 40-90

  • Make sure you do have some version of a window manager installed, apt-get install icewm should do.

  • dpkg-reconfigure xserver-xfree86 should open up an interactive configurator, try with it.


5.3.11. Window managers

Now you have X window system running. Let's make this clear: You need the X server because it knows how to communicate with your hardware and actually display graphics. But that's all it does. How your interface really looks like depends purely on the 'window manager'. If you listened to me, you are probably running icewm now, but there are others (when you install them, they become the default or show up in wdm's login menu). Try wmaker, blackbox, afterstep, xfce or enlightenment. Also try twm and fvwm at least for historical reasons, to understand the Unix folklore ;)

If you install GUI environments like Gnome or KDE, you won't have to worry about window managers as they will aready be taken care of.

Don't be disappointed by the look of wdm or icewm (tastes difer), you have plenty of other variants to choose; Definitely check out the Window Managers for X website.

To get a program which shows you graphical login (so you don't have to log in the console and type startx every time), install package wdm (or any of its relatives; xdm, kdm or gdm). Also, you will be able to select which window manager to use from the wdm's menu.

You can also run X without the window manager (usually for testing purposes). Try starting xinit.


5.3.12. Fonts for X

Fonts you might want are found in xfonts-* packages. Type this command to search for them:

# apt-cache search xfonts-

If you are interested in using the Microsoft ttf fonts, there are font servers which can handle them, and I'd recommend you try xfstt. No fonts come with it since they all have non-free licenses. That means you have to get the ttf fonts yourself, copy them to /usr/share/fonts/truetype/, add FontPath “unix/:7101” to /etc/X11/XF86Config, execute /etc/init.d/xfstt restart and then restart X.

To browse installed fonts, see the xfontsel and gfontview programs.

Also, check out the http://www.linuks.mine.nu/fonts/readme file.

Actually, X4 can deal with TrueType fonts directly, you don't need the ttf-enable font server; simply add the ttf FontPath in the X config file.

TODO: find the fontpaths for all xfonts- packages and list them here


5.3.13. Gpm (the console mouse driver) and the XFree86

You will most probably have problems with gpm and XFree86 running at the same time. The solution would be to set repeat type to 'raw' in gpm's config and mouse device to /dev/gpmdata in X config file, but that doesn't always give usable results. I prefer to shut down gpm.


5.4. Virtual consoles

5.4.1. Virtual consoles setup in /etc/inittab

Almost all GNU/Linux distributions ship with predefined 'virtual terminals' - completely separate text screens or consoles which are available with left Alt + F1-F6 keystrokes (only 6 consoles are enabled by default). You can also use the command-line method to switch between them (see the chvt command), and you can open them automatically with the open command.

To add more virtual consoles, edit the file/etc/inittab (as the superuser, of course) and add more lines like those:

5:23:respawn:/sbin/getty 38400 tty5
6:23:respawn:/sbin/getty 38400 tty6

[You can see which fields have to be incremented]. For changes in that file to take effect, exit the text editor and type init q.

If you create more than 12 consoles, you won't be able to access them with left Alt (since the last F key you have is 12), so use right Alt key to reach consoles 13 - 24. You can also use Alt + left_arrow or right_arrow to cycle through open consoles. Alt+print_screen key switches between two last used virtual consoles.

If you are switching from X to the console, you need to use Ctrl+Alt, instead of just Alt.

The deallocvt command frees memory still associated with virtual terminals which are no longer in use [by applications, not you of course]. This is not so important anymore, since you probably have plenty of ram and few kilobytes mean nothing to you.


5.4.2. VGA fonts sizes in the console

If you don't like such big letters in the console, execute this:

# lilo -R 'linux vga=ask' ; reboot

This would set up LILO parameters on the next boot (linux vga=ask), and reboot the machine (since vga mode can only be set at boot, unless you mess with 'svgatextmode' package - but don't do that). When you find a nice vga mode, you should edit /etc/lilo.conf and make it permanent there:

image=/vmlinuz
label=Linux
read-only
append="vga=X"

[X is replaced with the actual value you like, try '6' for example]. Then, run 'lilo' to apply changes.

If you see the penguin in the upper left corner of your screen, you are using a framebuffer (VESA mode). In that case, there are more screen modes available to you, see the table on the Framebuffer HOWTO page.


5.4.3. Font types for the console

Install the fonter package and you will be able to edit/create your own fonts, or use some of the standard ones you get:

$ consolechars -f /usr/share/fonter/crakrjak.fnt
$ consolechars -f /usr/share/fonter/elite.fnt
$ consolechars -f iso01.f16


5.4.4. The console keymaps

To see current keyboard mappings, you would simply do:

$ dumpkeys > keymap

After you tune the 'keymap' file to your needs, load it back with the loadkeys command.

To see just how advanced the idea of the Linux console is, run the loadkeys program, and type the following in its prompt:

string F1 = "Hello, World!"
[Ctrl+d]

Then just press the F1 key to see the consequences.


5.5. Shutting down the system

Some of the commands you can use:

# shutdown -h now
>

To reboot:

# shutdown -r now
>

Sometimes the shutdown -c (shutdown cancelation command) comes handy.

You can also use Ctrl+Alt+Del (in the console) to reboot, and this behavior is controlled from /etc/inittab.


5.6. General notes for hardware support

Getting a piece of hardware to work is a fairly easy task (although it wasn't so in the past, so always show the due respect for the developer community). Basically, you have to be able to categorize the hardware and know how the specific devices are usually configured under Debian GNU or Linux.

  • Hard disks and CD Roms are supported by the kernel.

  • Sound and network cards are supported by the kernel. Audio settings are saved in a mixer program's config (aumix, for example), and the setup information for the network is in the /etc/network/interfaces file; see interfaces(5) manual.

  • Graphic adapters can use the kernel support (mainly for AGP utilization and true hardware acceleration) but the drivers are usually the XFree server modules. You configure them by invoking dpkg-reconfigure xserver-xfree86.

  • Mice are supported in user space, by the console gpm package, or the graphical X server. The kernel, however, needs to support the port the mouse is connected to (serial, PS/2, USB ...). Use gpmconfig or dpkg-reconfigure xserver-xfree86.

  • Modems (standard hardware modems, not winmodems nor their recent HCF and HSF camouflages) require no special drivers, they are supported by the kernel serial driver which is almost certainly already active on your machine. If you have a winmodem (or those new HSF or HCF things), just forget it (supporting things which are "bad by design" doesn't make much sense in the Debian world). Configure the dial-up account with the pppconfig utility.

  • Scanners, digital cameras and USB devices use either kernel or userspace "drivers" (depending on the model and driver design), but the kernel needs to have basic support for them included.

Debian GNU sports a nice tool for kernel module configuration - the modconf utility. However, the whole story with kernel modules is trivial. You have three basic commands (modprobe, rmmod, lsmod) and a bunch of modules in the /lib/modules/`uname -r` directory to choose from. For instance, to load the driver for a 3Com network card and an onboard AMD VIA audio chip, you would do (so, without modconf):

# modprobe 3c59x
>

And to make the modules load at each boot, you'd add them to the /etc/modules file.

If you want to use it this way, you must know the module names. Until you get some experience, use modconf.

NotePlease Note:
 

If you are planing to compile your own kernel, definitely use the kernel-package helper (specifically, the make-kpkg command), which will save you a lot of trouble. Kernel compilation procedure now has its own chapter inside this guide, we'll come to it.


5.7. Enabling the mouse in text consoles

It is nice to have a mouse working in text consoles; you can copy just by selecting the text, and you can paste with buttons 2 or 3.

Install the gpm package and it will automatically ask you for configuration. If you want to delay it, or you don't get it right the first time, you can always re-run the config tool later (it's called gpmconfig).

Here's an example for you: for my wheel mouse, I answered this to gpmconfig questions:

  • Where is your mouse? /dev/psaux (that's PS2 port, use /dev/ttyS0 or /dev/ttyS1 for serial ports 1 and 2).

  • What type is your mouse? imps2 (most mice work with imps2 or ps2. Try 'ms' or 'bare' for serial mice)

  • responsiveness? *leave empty*

  • repeat protocol? use 'none'

  • additional arguments? *leave empty*

Test the config and enjoy.


5.8. Hard disk throughput

To see how good can it be, use the hdparm utility, switch to 'single' mode and test it:

# apt-get install hdparm
>

On ~1 Ghz PC machines, you should see cache reads of about 180 MB/sec (although this number has virtually no limit, on newer machines you get 500 MB/sec in a blink), and unbuffered disk I/O of about 30MB/sec on IDE disks (unless you're lucky enough to have those new and shiny 70MB/s IBMs).

Things vary, though. If you see poor performance (it can get as low as 2MB/sec), recompile the kernel and test it after that; you'll most probably see enormous improvements. Another great speed improvement comes from enabling DMA, say:

# hdparm -d1 /dev/hda

You can add the above command near the end of the /etc/init.d/bootmisc.sh and it will be re-activated on every machine boot (which is what you want). To sum up, unbuffered transfers of 25 MB/sec or more are okay for the traditional PC IDE disks.

If you feel lucky, use hdparm and try to fine-tune the hard disk parameters even further; see if it does any good for you. Once you're fine with the performance, remember to adjust the line in /etc/init.d/bootmisc.sh.


5.10. Firewalls

5.10.1. What is a firewall

That question would wave made little sense a few years ago (before 1997) but it seems to be a must today, when most computer-related things are just dumbed down and hidden behind graphical interfaces, and children waste their time practicing skills they have no or little use for in the real world.

Anyway, on to the subject. Running an Unix machine involves a great deal of responsibility, especially today when people have high-speed Internet connections at their homes; Unix systems don't basically make a difference between physically local and remote users. Anyone who gains access to your machine (especially to privileged accounts) can use it to compromise you and other hosts on your network or attack other Internet sites and cover his tracks. Depending on the type and success of the attack, sometimes the only solace you have is the physical access to the machine and the ability to reinstall it (let alone the backups you didn't make).

Therefore, we will now introduce you to firewall software:

A firewall is a set of related programs, located at a network gateway server or the user's machine, which protect the private resources from unauthorized [ab]use. Basically, a firewall examines each network packet to determine whether to forward it toward its destination. A firewall is often installed in a specially designated computer separate from the rest of the network so that no incoming request can get directly at private network resources.

This means we will use a firewall to control access to our machine, keeping in mind that we distinguish connections initiated by us, and those initiated by the remote ends.

CautionCaution
 

Installing and (mis)configuring a firewall is by no means enough to enforce the site usage policy or provide a satisfying level of security, but it does make a big difference compared to a vanilla ('out of the box') system (having a car doesn't make you a driver, but it solves a mandatory pre-requisite).


5.10.2. Firewall setup in Debian GNU/Linux

Free software firewalls have evolved. The old Linux 2.0 kernel series used ipfwadm, 2.2 had ipchains and the current stable 2.4 branch sports the shiny netfilter, sympathized even by those who always preferred BSD systems for that part of the job.

The user-space part (for netfilter) is covered by the iptables package, which is a rather low-level interface to the firewall functions so some people (yes, we too) tend to use frontends; I found ferm to be The Frontend. ferm is a 'firewall rule parser for linux designed to maintain and setup complicated firewall rules'. Fair enough.

# apt-get install ferm

We will now see what a generic home-firewall setup looks like. The policy we will follow is: drop everything, permit only port 113, manually specified IPs and traffic initiated by our side. You should read ferm man page and the examples in /usr/share/doc/ferm/examples/, but here's my suggestion to get us going:

# /etc/security/ferm.rules

option automod
option iptables
option clearall
option createchains

chain input policy drop;
chain output policy drop;
chain forward policy drop;

chain output accept;
chain input proto icmp accept;

chain input if ( eth0 lo ) {
(1)	saddr 192.168.7.110/24 accept;
	saddr 127.0.0.1 accept;
        drop log;
}

(2)chain input if ppp0 {
        saddr 129.70.28.189 ACCEPT;
        saddr 161.53.41.91 ACCEPT;

        proto tcp dport 113 ACCEPT;

        state (established,related) ACCEPT;

        drop log;
}

(1)
The example assumes your machine has the local IP address 192.168.7.110 and netmask 255.255.255.0. Adjust the host IP (netmask is probably okay).
(2)
The example assumes your Internet link is a dial-up connection ppp0. Adjust according to your setup.

TipTip
 

If you use dport or sport options in your rules, you must also include the proto tcp or udp specification.

To make the rules active:

# ferm /etc/security/ferm.rules

You could also add this command to the /etc/ppp/ip-up script, to have it start automatically, whenever the dialup link goes up.


5.10.3. More protection

Unless you are playing games under Wine or WineX, you could be interested in applying the grsecurity patches to your kernel (see apt-cache search grsec).

You could also install the Prelude Hybrid IDS (Intrusion Detection System) on your machines.


5.11. Setting up IP Masquerading/NAT

Multiple computers can all share the single connection (to the Internet usually) installed on the gateway machine. The procedure to set it up is trivial:

  • On the 'server' machine

# apt-get install ipmasq

  • On client machines

# route add default gw [server.ip]

To make client side changes permanent, add 'gateway' option to the /etc/network/interfaces file. Also, make sure the /etc/resolv.conf files on client machines are valid (copy from the main machine would do if you substitute references to 127.0.0.1 with the server's IP as it is seen from the local network).

NotePlease Note:
 

If it doesn't work for you (you get 'Operation not permitted' errors even on the server machine), try '/etc/init.d/iptables stop').

For laptops, or computers which often change their network environment, see the divine package.


5.12. System login procedure, the shell startup and config files

5.12.1. The system getty

We've mentioned the /etc/inittab file before. During the system boot, the init process (it always has the PID 1, it's the first process the kernel runs) reads that file and (among other things) initializes the virtual consoles, usually by starting the getty program on them. The system getty opens up a login prompt on the specified consoles and waits for users.

When you enter an username and password, your authentication request reaches the PAM layer (Pluggable Authentication Modules), where it gets checked for validity (using the /etc/pam.d/login rules); the check usually includes reading the /etc/passwd, /etc/group and /etc/shadow files and verifying the user's password and expiration dates. Please note that we are talking about the defaults here, the PAM system has endless configuration options, and it wouldn't be hard to make it use the retina scan instead of passwords to authenticate users.

The PAM was originally developed at Sun Microsystems, but the Linux people maintain a fairly compatible Linux-PAM tree. For the complete Linux-PAM user, administration and developer manuals, see the PAM documentation at kernel.org FTP site (the documentation is not on www.tldp.org).

NoteThe hint:
 

The getty does wait for your login, but if you do not authenticate successfully the first time, the next login prompt you would see would not be served by getty, but by the /bin/login program itself. They look almost the same, but I thought you'd appreciate this detail. You would see the original getty again either when you finish your work and log out, or you reach the maximum login retry limit, in which case the /bin/login would terminate and the init process would spawn another getty on that console.

If you press Enter on the empty console login prompt and it immediately serves you a new one, you know you're talking to the system getty. The /bin/login program would wait for a timeout instead, then tell you the login is incorrect.


5.12.2. The login shell

If the PAM layer gives you a green light, the login program spawns a shell for you (exactly which shell is specified in the last field of your /etc/passwd record). The shell then:

  • Executes /etc/profile and checks some other files (/etc/environment for example)

  • Executes the ~/.bash_profile dotfile, or ~/.profile if the former doesn't exist

  • Finally, gives you the shell prompt

NotePlease Note:
 

The root user does NOT read the /etc/profile and it's dotfile is /root/.profile. It's just a convention, the root's dotfile name is not exactly enforced by the system, ~/.bash_profile would have priority if present.

Also, the ~/.bash_profile is parsed only if you use the bash shell (check out the last field in 'getent passwd $USER'). If your shell is /bin/sh or something else, only the ~/.profile file will be read (if it exists at all).


5.13. Regular user accounts

If you are logged in as root, create a new regular user account with the 'adduser' command and reopen this guide in it. To illustrate why using root account for user tasks is strongly discouraged, I will quote a good summary by Debian users on IRC channel #debian@OPN:

It has been said that root is the administrative account - only use it when root power is needed. So no reading mail, compiling programs, or running applications as root. And don't even think about irc'ing as root, it increases the danger from exploits and trojans (such as bliss).

If you visit #debian on irc.openprojects.net, and people see you are logged in as root, they will most definitely harass you about it.

You should always be logged in as a regular user, and change current user ID (to root) only when necessary, using the 'su' command (or install advanced control mechanisms, such as 'sudo').


5.14. Switching to root account without a password

However, the problem is that you always have to type in the root password when you want to 'su' to root. To avoid this (that is, to enable 'su'ing to root without a password), edit the /etc/group file, and insert this line (anywhere):

wheel:x:28:username1,username2

and in /etc/pam.d/su, uncomment this line (remove the # char at the beginning, or copy this line there if you don't have it):

auth sufficient pam_wheel.so trust

That will allow users named 'username1' and 'username2' to type 'su' and become root without a password. Also, it will allow them to start processes as root on command-by-command basis with su -c '/command/to/execute with arguments'.

TipTip
 

To test it for the first time, completely log out and then log back in to reinitialize user groups information.

CautionCaution
 

Note that once you do it, the system security depends on username1 and username2 account passwords.


5.15. Account login regulation

Since most of the accounts on your machine will be used locally by you, you don't want people logging in remotely, do you? (they first need an account password for that, but they might get it easier than you think). Edit file /etc/security/access.conf, read short info there and add something like this to the file:

-:username1 username2:ALL EXCEPT LOCAL

This denies login to username1 and username2 accounts from all locations except your own machine.

TipTip
 

To make sshd obey the same restrictions, you need to put "UseLogin yes" directive in the /etc/ssh/sshd_config file and restart sshd (/etc/init.d/ssh restart). Actually, you deserve the whole story here: to minimize security risks, the portable version of OpenSSH uses the "privilege separation" mechanism which is enabled by default. Privilege separation, on the other hand, breaks PAM (if you ever wondered why /etc/pam.d/ssh doesn't work as you expected). So by saying "UseLogin yes" here, we put /etc/pam.d/login in effect (inconsistent and dirty, but it works for the moment).


5.16. Tcp wrappers

Tcp wrappers are a standard part of Debian, and allow you to simply control access to system services (mostly to those started from the inetd meta daemon). If you want to deny all services to remote addresses, make sure the file /etc/hosts.allow is empty, and put this in /etc/hosts.deny:

ALL: ALL EXCEPT LOCAL 127.0.0.1: DENY

For more information (including on how to trigger system commands upon incomming requests) read hosts_access(5) and hosts_options(5) man pages.

NotePlease Note:
 

Tcp wrappers and a firewall have very little in common; the level at which the allow/deny decision takes place is fundamentally different. With a firewall, it happens on a lowest, packet level: the packet targeted at say, an ftp port, could be dropped by the firewall as soon as it gets received by the network hardware and processed by the operating system's network layer (it wouldn't even reach the ftp daemon). With tcp wrappers, the packet does reach its destination (or the inetd, at least). The request validity check is usually performed before the server forks a new child process to service the incoming request.


5.17. Manually unpacking .deb files

From time to time you wish to unpack a .deb file to see its contents (or to recover some system files). Fortunately, Debian's .deb files need no special tools to be unpacked, they are simple 'ar' archives containing two files: data.tar.gz and control.tar.gz. Here are some examples:

  • Using dpkg to unpack the contents of a .deb file to an arbitrary directory:

    $ dpkg -x package.deb /tmp/package

  • Using 'ar' to unpack the data tarball:

    $ ar x package.deb data.tar.gz

  • Using 'ar' to unpack the control tarball:

    $ ar x package.deb control.tar.gz

NotePlease Note:
 

If you're not careful when upgrading/downgrading the gnu libc package on your system, you'll most probably loose the /sbin/ldconfig command, and most of the things you try to do will fail for that reason. If that's why you are reading this, then one solution is to unpack the libc6 package manually and copy the ldconfig command back in place; the other thing you can do is to create an empty ldconfig, which would simply return success:

# echo "#!/bin/sh" > /sbin/ldconfig
>


5.18. Checking the MD5 sums of installed Debian packages

It is often useful to verify the files on your system, either to detect unauthorized modifications or just to find out which files you once modifed and then forgot about them.

# apt-get install debsums
>


5.19. Shared sessions

Terminal sessions shared by more than one concurrent user can be very useful. On a few occasions, I was asked to remotely tune machines (such as sound card drivers or XFree86 support), and the other party wanted to keep a complete track of my actions (for educational and controlling purposes).

It is possible to achieve that effect by using either screen or ttysnoop.


5.19.1. screen

Using screen to make shared sessions is very easy, but it requires both parties to cooperate (so you must trust the other end) and involves shared account passwords (which is a bad thing if it becomes your habbit). All one must do is to login as say, 'username1' (ssh -l username1 localhost) and run 'screen', then wait for the other party to log on to the system (under the same username, of course) and run 'screen -x'.

(This tip was provided by electr0n@OPN).


5.19.2. ttysnoop

ttysnoop is a trivial but very convenient tool that can be used to share, monitor or control user terminals. Enabling ttysnoop on your machine is dangerous; it could violate your security policy or leave the system in an unusable state if not done properly. The ttysnoop itself doesn't need any special setup (except the /etc/snooptab file maybe) if both parties cooperate (one starts the ttysnoops server, and the other starts the ttysnoop client). However, installing it so that the ttysnoops gets started during the login does require a few changes in the system configuration files.

We will show here how to replace the system's login binary with ttysnoops and how to enable it for ssh connections. The procedure is delicate, as we said already, so we will comment each line you are about to execute in your shell.

CautionCaution
 

The /bin/login file, an important part of every Unix system, will get modified. This means that all applications which use /bin/login will be affected; in other words, it would become possible for users who posess the root password to completely monitor and control those character data streams (with the root password they could do it anyway, but not *so* easily). You shouldn't notice any visual changes, but please understand that the ttysnoop server will hook itself between the login program and the user (/dev/ttyp*). If you want specific services not to use the snooped /bin/login, instruct them to use /bin/login.real as the login program (that's exactly what we will do with the system getty).

(1) # dpkg-divert --divert /bin/login.real --add /bin/login
(2) # mv /bin/login /bin/login.real
(3) # echo "* socket login /bin/login.real" > /etc/snooptab
(4) # cp /etc/inittab /etc/inittab.valid
(5) # perl -p -i,orig -e 's#getty#getty -l /bin/login.real#g' /etc/inittab
(6) # ln -sf /usr/sbin/ttysnoops /bin/login
(7) # init q
(8) # echo "UseLogin yes" >> /etc/ssh/sshd_config
(9) # /etc/init.d/ssh restart	

(1)
We already described the dpkg-divert command above. It "diverts" the file /bin/login to /bin/login.real, meaning that new packages which contain /bin/login file will unpack it to a different location, /bin/login.real. To undo this step, use dpkg-divert --remove /bin/login.
(2)
Move /bin/login to /bin/login.real. The system login will be corrupted till step 6, when we re-create the /bin/login file. To undo this step, use mv /bin/login.real /bin/login.
(3)
Create the /etc/snooptab file, which contains a single rule "* socket login /bin/login.real". See man ttysnoop(8) for details.
(4)
Create a copy of the /etc/inittab file in /etc/inittab.valid. This is important; if anything bad happens to /etc/inittab you could end up with an unusable system, so having a valid copy lying around is encouraged (also leave one shell opened, so that you can put the valid file back in place even if you break system login).
(5)
Using Perl, edit the file /etc/inittab in-place, and replace every occurence of 'getty' with 'getty -l /bin/login.real'. The copy of the original file is saved in /etc/inittab,orig. *Never* run this command twice before putting the ,orig file back first (or you'll end up with something like 'getty -l /bin/login.real -l /bin/login.real'). In case of trouble, copy the .valid file from the previous step onto /etc/inittab. Also, note that we use 'getty -l' (where -l is smallcaps -L, not the number -1).
(6)
We re-create the /bin/login, making it a symbolic link to /usr/sbin/ttysnoops, the ttysnoop server.
(7)
Reload the init process, which re-reads the /etc/inittab file. If you made a mistake in some of the previous steps, your local consoles probably won't work anymore; that's why we suggested to leave one shell open and have a copy of the original /etc/inittab. If you decide to put the old inittab back, don't forget to move the login.real file back too and remove the divert.
(8)
We append 'UseLogin yes' to the end of the sshd configuration file.
(9)
We restart the sshd daemon.

WarningWarning
 

Enabling ttysnoop on your machine is dangerous; it could violate your security policy or leave the system in an unusable state if not done properly. For example, if you loose the ability to start X as a regular system user, chances are you did not make getty use the original login program so either fix that, or run dpkg-reconfigure xserver-common and allow anyone to run X server (a bad thing to do).

You can test the setup locally (but the same idea applies to remote logins, of course):

  • ssh to your localhost (execute: ssh 127.0.0.1 or ssh 0, which works on Linux only)

  • switch to another virtual console (or X terminal) and login as root. Find out the correct tty device (ttyp*) for our snoop target:
    # w | grep ttyp
    myuser ttyp0 - 4:20am 3.00s 0.05s 0.02s -bash 

  • invoke the ttysnoop to hook to /dev/ttyp0:
    $ /usr/sbin/ttysnoop ttyp0

  • type in root password (to authenticate with ttysnoops) and enjoy your shared view ;p

When letting people log in remotely to your machine, ssh is strongly-preferred way to connect. Do not even bother with telnet (which is an unencrypted and insecure service). If you have special needs or demand telnet anyway, check out working configurations from the sample /etc/snooptab files.


5.20. Runlevels and system services

5.20.1. System boot and the init process

This is a very interesting and important part of every Unix system.

In most common scenarios, you have LILO installed as the bootloader. LILO (the LInux LOader) accepts parameters on the command line, but Debian has been configured (in default configuration) not to show the LILO boot prompt. To make it appear, hold the Alt key at the 'LILO' message (during boot, just before you see the 'Loading linux ....' message) and you'll be able to pass arbitrary parameters to kernel. You can type anything there, and it will later be visible in the /proc/cmdline file.

After the kernel gets loaded, it starts 'init' as the first system process. Init executes the tasks defined in the /etc/rcS.d directory. Init then enters default runlevel 2 (other Linux distributions mostly use runlevel 3 as the default) and executes the tasks defined in the /etc/rc2.d/ directory. Init directories consist of symbolic links to files in /etc/init.d/; here's an example:

$ ls -la /etc/rc2.d/ | cut -b 57-
...
S20net-acct -> ../init.d/net-acct
S20openldapd -> ../init.d/openldapd
S20postgresql -> ../init.d/postgresql
...

The 'S' prefix starts a service, while 'K' stops it (for the given runlevel). The numbers determine the order in which the scripts are run (0 being the first).

init then excutes local scripts from /etc/rc.boot/ and performs the rest of init tasks specified in /etc/inittab.


5.20.2. Manipulating /etc/rcX.d/ links

Debian provides a convenient tool to manage runlevels (to control when services are started and shut down); it's called update-rc.d and there are two commonly used invocation methods:

# update-rc.d -f xdm remove
>

The first line shows you how to remove the xdm service from startup; the second sets it back. xdm is a basic implementation of the graphical login screen. You could disable it if you prefer console logins (although nothing is stopping you from using both at the same time), or you could try xdm alternatives, such as kdm, gdm or wdm.


5.20.3. Manual service start/stop

All files in /etc/init.d/ share a common invocation syntax, which is defined by Debian Policy. All system services have their init script there (usually named as the service itself) which accepts generic arguments. Let's see an example:

# ls -al /etc/init.d/a* | cut -b 55- 
/etc/init.d/acct
/etc/init.d/apache
Starting web server: apache.
/usr/sbin/apachectl start: httpd started
Stopping web server: apache.
/usr/sbin/apachectl stop: httpd stopped
Usage: /etc/init.d/apache {start|stop|reload|reload-modules|force-reload|restart}

NotePlease Note:
 

  • A generic init.d script template is in /etc/init.d/skeleton, use it for your own scripts. Also check the /etc/init.d/bootmisc.sh file.

  • Besides the traditional System V init system, Debian also supports single-file based init. See the file-rc package if you're interested.


5.21. The Debian 'alternatives' system

The Debian distribution comes with a lot of software packages and chances are you can choose between a few different applications that basically perform the same task; you can even have them all installed and peacefully coexisting on the system. Since we speak of Debian, there intuitively has to be an elegant way to set system defaults; and yes, there is one: the 'alternatives' system. Here's a similar introduction and an example from the update-alternatives(8) man page:

It is possible for several programs fulfilling the same or similar functions to be installed on a single system at the same time. For example, many systems have several text editors installed at once. This gives choice to the users of a system, allowing each to use a different editor, if desired, but makes it difficult for a program to make a good choice of editor to invoke if the user has not specified a particular preference. Debian's alternatives system aims to solve this problem. A generic name in the filesystem is shared by all files providing interchangeable functionality. The alternatives system and the system administrator together determine which actual file is referenced by this generic name. For example, if the text editors ed(1) and nvi(1) are both installed on the system, the alternatives system will cause the generic name /usr/bin/editor to refer to /usr/bin/nvi by default. The system administrator can override this and cause it to refer to /usr/bin/ed instead, and the alternatives system will not alter this setting until explicitly requested to do so.

To actually configure the mentioned 'editor' alternative, simply type:

# update-alternatives --config editor

For a list of possible alternatives, check out the /etc/alternatives/ directory. You will most likely want to tune the 'x-window-manager' and 'x-session-manager' choices.


5.22. Periodically checking for the available disk space

It is very important to monitor disk usage. If the disk becomes full (especially on your /home partition), you will probably damage your config files or even lose data. XFree86 won't even start if there's no free disk space available. A simple crontab script which would just send you a daily disk usage reminder could be created this way:

# echo '#!/bin/sh' > /etc/cron.daily/disk-usage
>

Make sure to then edit the /etc/crontab file to adjust the time at which the cron.daily/ parts are run (it defaults to 06:25 AM).


5.23. Creating and extracting file archives

In Unix, you traditionally group ('tar') many files into a single one, and then compress it using a compression alghoritm (gzip for example). We will show here how to create and extract tar, tar.gz and tar.bz2 archives.

# apt-get install bzip2







>

For zip, rar, arj or cab files support, use the apt-cache search command to find the packages you need to install.


5.24. Copying, mirroring and re-downloading Debian packages

5.24.1. Re-downloading all installed .deb packages

You might want to do that for mirroring or backup purposes, or to burn a CD set with satisifed dependencies. The trick can be achieved with:

# COLUMNS=200 dpkg -l | grep '^ii' | awk '{ print $2 }' > /tmp/pkglist
>

After the download finishes, you should see all the .deb files in /var/cache/apt/archives/. Also, see the apt-move sync method below, maybe that's what you really need.


5.24.2. Setting up a Debian mirror or an apt source on the local hard disk

This subsection could be of interest to you if you want to copy Debian CDs to your disk (so you don't have to change cdroms in the drive all the time) or export the Debian mirror to the machines on your local LAN.

The easiest way to copy Debian CDs to your hard disk would be to use the dd tool and directly create CD images:

# dd if=/dev/cdrom of=/mnt/deb-cd1.img bs=512k

TipTip
 

If you install the cdrecord package, the readcd command will be available to you:

# readcd dev=0,0,0 f=deb-cd1.img

You would then mount the image like this (of course, you add it to /etc/fstab if you want it permanent):

# mkdir /mnt/deb-cd1
>

An example sources.list entry would look like this:

deb file://mnt/deb-cd1 woody main contrib non-free non-US/main non-US/contrib non-US/non-free

NotePlease Note:
 

While this procedure would work and is perfectly fine if you really want the CD images (if you are, say, recording Debian CDs), you'll probably find it inconvenient for other purposes (because the kernel default maximum number of loop devices in only 7 and you will necessarily have multiple sources instead of one which has everything).


5.24.3. Using apt-move to move packages into the local hard disk repository

If you have enough disk space, you'll find apt-move very convenient. You can use it to copy the cdroms to the disk, sort the downloaded files from /var/cache/apt/archives, or stay in sync with your local Debian mirror.

  • First, install apt-move:

    apt-get install apt-move

  • Edit /etc/apt-move.conf, put 'localhost' (or the Internet mirror name, of course) under APTSITES field and change the DIST option (to 'woody', if you have Debian 3.0). You could also change PKGTYPE to 'both' if you also want source packages.

  • Edit /etc/apt/sources.list and deactivate all lines referring to cdroms.

  • Export /cdrom with Apache (if using http://localhost instead of the Internet mirror as package source):

    # apt-get install apache
    >

  • Initialize apt-move:

    # apt-move get

  • Now for each CD you wan't to move to your hard disk, type:

    # mount /cdrom
    >

    If you don't use CDs but the Internet mirror directly:

    # To only move packages from /var/cache/apt/archives/
    
    
    >

  • When you are finished, create the new Packages.gz and Sources.gz:

    # apt-move packages
    >

    Note that the gunzip .. line is important, because it creates the plain Packages files; you must have them if you're later using tools like debootstrap. Also, you will need to copy some existing Release file to the dists/woody/ directory.

  • Replace the http://localhost/cdrom/ repository (we don't need it any more) with the newly created mirror and install xplanet to test it:

    # perl -n -i -e 'print unless m#/cdrom/#' /etc/apt/sources.list
      >> /etc/apt/sources.list
    >

    Before you move on, don't forget to remove the /cdrom/ entry from /etc/apache/httpd.conf.


5.25. Package recompilation

Debian software packages come precompiled for many architectures. In the case of Intel-compatibles, those packages are optimized to work with all variants from i386 up. However, it means that all the system binaries you have use nothing from the advanced features found in modern processors (Pentiums for example).

There's been much of a debate if local recompilation of packages would give any advantage, but the general (and sane) conclusion seems to be this:

Local recompilation makes sense for kernel source, the GNU C Library (glibc), compression tools (such as gzip or bzip2) and some open-source games.

To recompile and install the package (say, vim), use:

# apt-get -b source vim


5.25.1. Building .deb packages from source

TODO: source_builder.pl, apt-get --build, cast

fakeroot dpkg-buildpackage -uc -us

Installing software from generic .tar.gz packages - the checkinstall project (apt-get install checkinstall)


5.25.2. dpsyco - Debian Packages of System Configurations

Dpsyco is an acronym for Debian Packages of System Configurations and you can create and maintain "configuration packages" with it. A configuration package is a package that resides "on top" of the normal debian packages. You can overwrite normal files, patch the system, add users and groups and much more.

# apt-get install dpsyco dpsyco-{skel,patch}


5.25.3. equivs - inform dpkg about localy installed packages

This is a package, that creates Debian packages that can be used to inform dpkg about localy installed packages and their dependancies. Also empty packages that just require other packages can be created with equivs. These can be used as "profile" packages that just mark other ones for installation.

Please note that this is a crude hack and if thoughtlessly used might possibly do damage to your packaging system. And please note as well that using it is not the recommended way of dealing with broken dependencies. Better file a bug report instead.


5.26. Linking to your local Internet Service Provider

5.26.1. Dial-ups

You don't have load any drivers to support your modem. It's done automatically, the generic 'serial.o' driver takes care for serial port communication, and all normal modems understand the AT command set. If you have a winmodem (braindamaged modem which lacks one $5 chip and its work is loaded onto machine's CPU), then just forget it (however, there is a LinModems website, if you don't want to listen the voice of sanity).


5.26.1.1. pppconfig

Type pppconfig (as root) to create a new connection and define configuration parameters. You'll probably get it right just following the menus, but let me give you some some guidelines:

  • Leave "provider" as the name of first connection you make (so it becomes the default).

  • Choose Dynamic DNS.

  • Authentication method: find out what method your ISP uses. Nowadays, PAP is almost always a safe bet.

  • Modem port speed, leave at 115200 if you use external modem, or set other value if it's internal (such as 57600).

  • Choose 'Yes' to automatically detect your modem, then accept the port it found (see below on what to do if it doesn't find any).

  • You are now on the main screen where you can modify selected options before saving them to disk. Choose 'Advanced options'. In Europe, you will probably have to change Modeminit field from 'ATZ' to 'ATx3l1m1' (use l0m0 if you dont want the modem speaker to be heared); the 'x3' is important here: if you don't set it, you will get a 'NO DIALTONE' error.

NotePlease Note:
 

pppconfig is nothing but a convenient interface to create /etc/chatscripts/<provider> and /etc/ppp/peers/<provider> files. You could create them by hand if you wanted, too.

Unlike other "dialer" systems, the pon/poff method uses the pppd's ability to call the remote end. The /usr/bin/pon was a shell oneliner just until recently, when some description and error handling were added.


5.26.1.2. pon, poff

To connect to your default provider, type pon.

To terminate connection, use poff.


5.26.1.3. Connection statistics

There are plog and pppstats from the ppp package. You could also install the 'pppstatus' package for a really nice real-time monitor.

NotePlease Note:
 

Before you try plog, make sure you add "local2.* TAB /var/log/ppp.log" to your syslogd's configuration; the exact instructions were given earlier in the document.


5.26.1.4. Troubleshooting

  • pppconfig doesn't detect your modem

  1. make sure it's really a Hayes/Rockwell compatible modem (winmodems are not) which understands the AT command set

  2. make sure the modem is connected properly to the machine, and try switching it off/on to reset it

  3. try manually specifying ports from /dev/ttyS0 to /dev/ttyS3 (com ports 1 to 4), maybe you'll be lucky.

  • ping 161.53.2.66 works, but ping www.google.com doesn't:

  1. if the connection seems to work (modem lights are on, etc..) but you can't ping using FQDN (why I never really liked this name?) - it means the automatic DNS setup failed. In that case, find out what's the DNS (Domain Name Service) IP of your ISP and edit the file /etc/resolv.conf. Make it look like this:

nameserver your.dns.server.ip


5.26.1.5. Alternatives to pppconfig/pon/poff

pppconfig, pon and poff are superior tools. However, you could try another program, wvdial. The advantage of wvdial is that it can automatically detect what type of authentication your ISP accepts (and if it doesn't succeed, try setting "Stupid Mode = 1" in /etc/wvdial.conf).


5.26.2. ADSL with pppoe

Surprisingly, the adsl+pppoe setup is trivial with Debian. Install the pppoe package, find out the exact name of your provider (by sending the PADI packet) and read last notes in README.Debian:

# apt-get install pppoe
>

NotePlease Note:
 

There's also the pppoeconfig program. Maybe you can get away with it.


5.26.3. I'm connected to the Internet, now what?

Make sure you have the ircii package installed (or some other variant of irc client, such as console irssi or graphical xchat and kvirc). Then connect to the IRC server (irc.debian.org, which is a part of the FreeNode network [former "OpenProjects"]) and join channel #debian:

# su -c 'apt-get install xchat'

/server irc.debian.org
/join #debian

You can ask for help and advice there, or just hang around and collect useful tips.

WarningWarning
 

I hope you noticed the su -c ' ... ' above; it allows you to execute apt-get install xchat as root. The implicit message is that you should not be using the root account on a regular basis.

NotePlease Note:
 

Consider visiting the The serious bit, The Parody Site, and the Petition page to understand the 'Global Notice' money-requesting messages you'll be getting if you spend some time on the OPN network.


5.27. The package popularity contest

Let's just quote the popularity-contest manpage which says it all:

The popularity-contest command gathers information about Debian packages installed on the system, and prints the name of the most recently used executable program in that package as well as its last-accessed time (atime) and last-attribute-changed time (ctime) to stdout.

When aggregated with the output of popularity-contest from many other systems, this information is valuable because it can be used to determine which Debian packages are commonly installed, used, or installed and never used. This helps Debian maintainers make decisions such as which packages should be installed by default on new systems.

Normally, popularity-contest is run from a cron(8) job, /etc/cron.weekly/popularity-contest, which automatically emails the results to Debian package maintainers according to the settings in /etc/popularity-contest.conf.

# apt-get install popularity-contest


5.28. Accessing data on MS Windows partitions

Type 'fdisk -l' and you will see the list of partitions on all disks you have in the machine. Ignore everything (including warnings) except partitions which are of type FAT16 (for older Windows) or NTFS (for that Neandertal Technology (tm) stuff or newer). See this example output:

# fdisk -l

Disk /dev/hda: 255 heads, 63 sectors, 1823 cylinders
Units = cylinders of 16065 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hda1             1        62    497983+  82  Linux swap
/dev/hda2   *        63        70     64260   83  FAT16
/dev/hda3            71      1823  14080972+   5  Extended
/dev/hda5   *        71       101    248976   83  NTFS
/dev/hda6   *       102       709   4883728+  83  Linux
/dev/hda7           710      1317   4883728+  83  Linux
/dev/hda8          1318      1823   4064413+  83  Linux

Disk /dev/hdc: 16 heads, 63 sectors, 29065 cylinders
Units = cylinders of 1008 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hdc1             1     29065  14648728+  83  Linux

We have two physical disks, which are called /dev/hda and /dev/hdc. Windows partitions we are searching for are named /dev/hda2 and /dev/hda5 (see? it says FAT16 and NTFS there). Now create /mnt/hda2 and /mnt/hda5 directories (using the 'mkdir' command) and edit /etc/fstab to make your changes permanent:

# mkdir /mnt/hda2 /mnt/hda5

/dev/hda2 /mnt/hda2 vfat defaults,ro 0 0
/dev/hda5 /mnt/hda5 ntfs defaults,ro 0 0

As root, type 'mount /mnt/hda2' and windows data will be there, under /mnt/hda2/. The same procedure applies to the hda5 partition.

By adding the above to /etc/fstab we ensured it gets mounted on every boot. If you don't want that, add the 'noauto' option to the Options field ('defaults,ro'). Also, replace 'ro' with 'rw' if you want full read-write access.

Also, to mount MS windows partitions, you need appropriate kernel module support, but Linux loads it automatically. See 'lsmod' output, you should see 'vfat' or 'nfts' there after you mount some windows partition.

TipTip
 

It is interesting to note that the mount command supports disk mounts by partition labels, so you don't need to know the device file names in advance, and they can change later, requireing no modifications on your side (see the mount(8) man page, and pay attention to the -L option).

# e2label /dev/hda7 LINUX_HOME
>

6. Linux processes

In this chapter we will explain the processes and the way they run under the Linux kernel. Furthermore, you will learn common methods to start, stop and control them.


6.1. Introduction

In order to get us clear on what the word "process" really means, I searched for a formal, fluent explanation from the WhatIs database:

A process is an instance of a program running in a computer. It is close in meaning to 'task', a term used in some operating systems. In Unix, a process is started when a program is initiated (either by a user entering a shell command or by another program). Like a task, a process is a running program with which a particular set of data is associated, so that the process can be kept track of. An application that is being shared by multiple users will generally have one process at some stage of execution for each user.

A process can initiate a subprocess, which is a called a child process (and the initiating process is sometimes referred to as its parent ). A child process is a replica of the parent process and shares some of its resources, but cannot exist if the parent is terminated.

Processes can exchange information or synchronize their operation through several methods of interprocess communication (IPC).

Processes in Linux work in isolated memory areas (called 'segments') and if a process starts behaving badly, it surely won't affect the rest of the system.


6.2. Basic process-related commands

6.2.1. Process lists

To see the full list of current processes on the system, type:

$ ps aux
$ ps auxwww

And the sample output would be:

USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
root       398  0.0  0.8  2872 1084 tty1     S    11:57   0:00 -bash
root       399  0.0  0.3  2136  412 tty2     S    11:57   0:00 /sbin/getty 38400 tty2
root       400  0.0  0.3  1616  412 tty3     S    11:57   0:00 /sbin/getty 38400 tty3
root       401  0.0  0.3  1484  412 tty4     S    11:57   0:00 /sbin/getty 38400 tty4
root       402  0.0  0.3  2292  412 tty5     S    11:57   0:00 /sbin/getty 38400 tty5
docelic   2850  0.0  0.7  2768  984 tty6     S    15:39   0:00 -sh
docelic   3114  0.0  0.6  2472  772 tty6     S    16:57   0:00 /bin/sh /usr/bin/X11/startx
docelic   3409  0.0  1.0  2332 1300 pts/2    S    17:20   0:00 -sh
docelic   3410  0.3  4.9  9048 6260 pts/2    S    17:20   0:04 gvim hands-on.sgml
docelic   3415  0.0  1.0  3032 1312 pts/4    S    17:21   0:00 -sh
docelic   3428  0.0  3.4  7524 4344 pts/4    S    17:24   0:00 gvim hands-on-guide,editing.sgml
docelic   3611  0.0  1.0  3040 1276 pts/0    S    17:43   0:00 -sh
root      3612  0.2  1.0  2648 1288 pts/0    S    17:43   0:00 bash
root      3614  0.0  1.2  3940 1604 pts/0    R    17:43   0:00 ps auwww

Column names are in the first row. A few explanations are in order:

  • USER: Owner of the process (this also indicates the privileges a process has).

  • PID: Unique Process ID. Process IDs are used to distinguish processes, and are usually given incrementally. The numbers are not reused until the PID reaches its maximum value (32768 mostly) and begins from the start.

  • CPU, MEM: Total time the CPU spent executing the task (notice this is the CPU time, not the real time), and the current ammount of memory occupied by the process.

  • TTY: The controlling terminal for a process (usually a place where the process was started from). If that field is empty (contains the ? character), it means the process has detached from its terminal, or wasn't even started in an interactive mode.

  • STAT: Current process state: SW = swapped, S = sleeping, R = running.

NotePlease Note:
 

  • The incremental order in which PIDs are selected can be changed; most of the system security patches modify this behavior (such as grsecurity).

  • It is possible to restrict the list of processes a person can see (usually to its own processes). Most of the system security patches will suggest you to do it.

  • It may happen that you see processes hung in states D or Z. The Z character indicates 'zombie' processes; those whose parent process didn't shut them down properly, (usually) due to its own abnormal termination. The 'D' indicates a process which is blocked in a kernel function call; such states are normal for a short periods of time, but if you see them in the ps output, you'd better check what's actually going on.


6.2.2. System, processor and memory information

6.2.2.1. uptime

The uptime command reports current time, machine uptime, the number of login shells and the machine load:

$ uptime
20:30:17 up  8:33,  5 users,  load average: 1.15, 1.19, 1.16

NotePlease Note:
 

The three numbers show load averages for the past 1, 5 and 15 minutes.


6.2.2.2. free

free displays the amount of free and used memory in the system:

$ free
             total       used       free     shared    buffers     cached
Mem:        126500     100780      25720          0       7624      49324
-/+ buffers/cache:      43832      82668
Swap:       497972      31796     466176

  • Mem: The total ammount of available, used and free system memory. The 'shared' field should tell you how much memory is shared among processes, but you will most probably see zero (0) there, since the interface has changed in 2.4 kernel series, and the field was left for compatibility reasons.

  • The Swap: line shows the same information for the swap space. A swap space is usually located on your hard disk and used as the virtual memory extension of a computer's real memory (random access memory). Having a swap space allows the kernel to pretend that you have more RAM than you actually do.

NotePlease Note:
 

  • When the system decides to release a part of RAM which is occupied by a system binary it does not put it in swap because the "cost" (in amount of time) to retrieve it directly from the filesystem again is almost the same.

  • Many have asked why does the system use so much RAM (see the 100MB in our sample output). The answer to this question is very short, but let's not miss the opportunity to gain some more insight:

    Looking at the memory installed in a computer, we make the following diagram:

    Hard disks are big (> 50 GB) but their access time is poor; RAM is faster and affordable in 512-1024MB ammounts; the processor cache is very fast but typically only 128-512KB big (on typical PC computers). Such structure is dictated by the cost of the components on the market.

    Now, the processor cache and the system RAM are just subsets of the hard disk memory (except for the runtime-generated data, of course). It is a total waste (and an agression on system performance) to delete data from the system RAM; the runtime cost to retrieve it again from the hard disk is high. Therefore, the system keeps it loaded for as long as possible. To see the realistic RAM usage, refer to the "-/+ buffers/cache:" column in the free output, which subtracts the buffer and cache sizes from the total ammount of used memory.

  • Commands to check out if you're interested in system swap memory are mkswap, swapon and swapoff (and the swap space is, of course, configured in the /etc/fstab file).


6.2.2.3. top

The top command is an interactive process monitor which shows system, processor and memory status, and the periodically updated list of most intensive system processes.


6.2.2.4. Other commands

Other interesting commands include sar, iostat and mpstat (from the sysstat package), vmstat (from the procps package), the graphical lavaps and native Gnome or KDE frontends.


6.2.3. Misc commands

Three commands we definitely can't miss are strace, lsof and fuser. Maybe you won't use them right now, but consider this as an investment for the future.

The strace tool traces system calls and signals; its output could be overwhelming for you, but that's where the manpages and practice come into play.

# strace ls

lsof is an extremely useful tool that will give you a list of all open files (on a system or process basis).

# lsof -p 1

Finally, the fuser command will tell you which processes keep the specified file in use.

# fuser -v /dev/tty1


6.3. Two distinct types of executable files

A process is, as we said already, an instance of a program. A program can either be a compiled (binary) file, or a script which needs an interpreter to run. Install the file package now, so that we can use the file command in our demonstration. The /bin/ls program is an example of a binary file:

$ file /bin/ls
/bin/ls: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped

$ vim /bin/ls
^?ELF^A^A^A^@^@^@^@^@^@^@^@
[we get garbage of course ]

Let's see what file says for some other system executables:

$ file `which dpkg`
/usr/bin/dpkg: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped

$ file `which apt-cache`
/usr/bin/apt-cache: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped

$ file `which pon`
/usr/bin/pon: Bourne shell script text executable

$ file /usr/bin/822-date 
/usr/bin/822-date: perl script text executable

Now we know that dpkg and apt-cache are compiled, binary executables as well. We also know that the pon command is actually a Bourne shell script and the 822-date is a Perl script. Try viewing them in a text editor.

From the Unix system perspective, there's no real difference between the compiled and script executables. All the files need to have the exec bit set to be considered executable. File modes and permissions won't be explained here, see the Debian User Reference Manual, section 4.4 for an explanation.

In addition to the exec bit, script files need to contain a "shebang" line, the special directive which specifies the intepreter to invoke if none was specified on the command line (in case you just type /usr/bin/822-date instead of perl /usr/bin/822-date, or /usr/bin/pon instead of sh /usr/bin/pon). Let's check out the first few lines from the 822-date script and verify our predictions (don't get confused here, we are interested in the first line only):

#! /usr/bin/perl --


@ARGV && die "usage: 822-date\n";

   .... ....

Very well. This section actually has the purpose to teach you the following three things:

  • Executable files are not necessarily compiled binaries; they can be script files too, and if the shebang line is present, they can be started the same way the true binaries are (that is, just by typing in their name).

  • Script files are always associated with a specific interpreter. You could invoke "/usr/bin/pon", as well as "sh /usr/bin/pon" ( [interpreter name] [script file] [arguments] ).

  • Compiled, binary files have a variant of an interpreter too, although most people are not aware of it. For example, to run the ls command using an "interpreter" (it's actually "the helper program for shared library executables"), you would do:

    $ /lib/ld-linux.so.2 /bin/ls

    This thing could come very handy; I, however, leave it as an exercise for the reader to find out exactly when and how it can be used. [Don't you just love when the documentation says "an exercise for the reader" at the point where it becomes the most interesting?]


6.4. How to start a process

  • Type the program name at the command prompt. This is okay if the program does its job quickly and then exits, or is a daemon process (which puts itself into the background). Otherwise you won't get your command line back, since the program will open the terminal as its STDIN (Standard Input) channel.

    $ ls

  • Type the program name followed by '&'. This is called a 'background process'. It returns the command prompt back to you, and the process resumes in background (or if it can't work in background, as it is the case with tee - it will wait till you resume it in the foreground).

    $ tee&

  • Use nohup to run a command immune to hangups, and with output redirected to a non-tty. Useful if you want to leave a task running on the system while you're gone (and logged out).

    $ nohup ls

  • Use nice to run a command with a specific (lower if you are not root) priority.

    $ nice -n 5 top

NotePlease Note:
 

If you try to run some of your own programs from the current directory just by typing their name, you'll most probably get the "Command not found" error [followed by my lesson on the PATH variable :) ].

PATH is an environment variable which holds the names of directories to search for executable files. When you type in the ls command, the system locates it (using the hints from the PATH variable) in the /bin/ directory and executes from there. Only few standard directories are included in the path, so you'll have to prefix your non-standard command with "./" to indicate you want the specific command in a specific (current) directory:

# Let's create a tmp directory, 'cd' into it and create a sample executable file to use in the demonstration
$ mkdir tmp; cd tmp
$ echo "echo Hello, World" > test.sh
$ chmod 755 test.sh

$ test.sh
sh: test.sh: command not found

$ ./test.sh
Hello, World

$ cd ..
$ ./test.sh
sh: ./test.sh: No such file or directory

$ ./tmp/test.sh
Hello, World


6.5. How to terminate a process

  • Wait for a process to terminate normally (when it finishes its job).

    $ who

    NotePlease Note:
     

    The who command has a very famous history, it originally appeared in 1967!

  • Switch to another virtual console, use the ps command to find out the Process ID, and kill it.

    # This will work unless the process is in a bad state or ignoring the regular kill signals
    $ kill [PID]
    
    $ kill -9 [PID]

    NotePlease Note:
     

    The kill command is actually used to send any signal to a process, not just kill; see the kill(1) manpage.

  • If the process is interactive (i.e. it holds your command prompt), use Ctrl+C or Ctrl+D to finish it. The difference here is that Ctrl+C is intended to cancel the process, while Ctrl+D indicates 'end of input', so Ctrl+D is sometimes used to the tell the program to stop reading input, and to continue processing it.

    $ tee
     [Ctrl+c]
    $

  • If you started a process with '&' at the end (and you're still on the same terminal), type 'jobs' to see the list of background processes. Then just type kill %[JOB]

    $ tee&
    [1] 5652
    
    $ jobs
    [1]+  Stopped(SIGTTIN)        tee
    
    $ kill %1
    [1]+  Terminated              tee

  • Use the killall command, to kill processes by name.

    # killall ssh

  • Use the pkill command to look up or signal processes based on name and other attributes.

NotePlease Note:
 

When the process terminates, you can see its exit status (that is, the exit status of the last executed command) by typing: 'echo $?'. Zero (0) indicates success; for other (error) values see the program documentation.

NotePlease Note:
 

Be careful with killall. Its behavior is not the same on all Unix machines.


6.6. How to put a process to sleep?

By sending a signal to it:

kill -STOP [PID]
kill -CONT [PID]

When you start a process in foreground, and it takes your prompt away, you can press 'Ctrl+z' to put it in background and stop it. Later you can issue bg or fg (to resume it in the background or foreground). You can even put it in the background, and then at some later point use fg to get back to it (but not after you close the controlling terminal, because logout disassociates the tty from the process).

You can also press CTRL+s and CTRL+q on a process STDIN channel (usually your terminal) to pause/resume it.


6.7. Process priorities

Linux (and other Unix) systems have a way to specify the priority of a process, which is (on Linux) in the range from -20 (absolute priority) to 19 (which runs when there's nothing else to run). The kernel process scheduler is a serious piece of software and it works as expected (unlike schedulers found in some non-Unix systems).

Regular users can only use values from 0 to 19 (and 0 is the default). To run a program with altered priority (for example, 'top'), do:

$ nice -n 10 top
>


6.8. Processes and their input/output functions

Generally, each process will open 3 communication channels:

  1. STDIN (Standard Input, 0) to retrieve the input data

  2. STDOUT (Standard Output, 1) to output data

  3. STDERR (Standard Error, 2) to print error messages

In a typical user session (just typing 'ls', for example), all three are open to your terminal - it's the place where you input information, and the place where the application outputs any info for you (including errors). However, those communication channels (or more formally 'file descriptors') may be opened to a socket, pipe or any other place you can think of (including as input to another program or remote computer).

Let's look at an example: (grep will search all files and print file lines which contain the string '/var/log/messages' in them):

$ cd /etc
$ grep '/var/log/messages' *

grep: shadow: Permission denied
grep: shadow-: Permission denied 
grep: skel: Is a directory
grep: sound: Is a directory 
grep: ssh: Is a directory 
grep: syslog-ng: Is a directory 
syslog.conf:*.warn;mark.*;auth,authpriv.none /var/log/messages 
syslog.conf,old: mail,news.none -/var/log/messages 
syslog.conf.syslog-facility-old:*.=info;*.=notice;*.=warn %classic /var/log/messages 
grep: t1lib: Is a directory 
grep: terminfo: Is a directory 
grep: texmf: Is a directory

Obviously, one part of output is valid data (reports from syslog.conf and syslog.conf,old), and the other part are error messages ('Is a directory' and 'Permission denied'). Now examine the following slightly modified commands and their descriptions:

# redirect the stderr channel to the /tmp/errdata file
$ grep '/var/log/messages' * 2> /tmp/errdata

$ grep '/var/log/messages' * > /tmp/outdata 2>&1

$ grep '/var/log/messages' * >& /tmp/outdata

$ tee < /etc/syslog.conf

$ ps aux | cut -d" " -f 1

$ ls -al | less

$ ls -al | grep root


6.9. Leaving processes running while you're away

You might want your processes to run while you are not logged in.

The first trick is to use nohup or '&' at program startup time (as we mentioned before).

If you already have it running, you might protect it with 'disown -h' (but you won't find the disown's man page, it's one of bash shell's built-ins, and is described in bash(1) manpage).

$ ./mycommand &
$ nohup ./mycommand2
$ nohup ./mycommand3 &
$ disown -h %1

NotePlease Note:
 

Note however that you won't be able to re-gain control of the process input/output functions once you log back in, you only ensure they are not closed along with your shell when you log out.

The superior way is to use the screen utility which allows you to resume complete sessions. Use Ctrl+a,d to detach and screen -r [ID] to resume a session. Familiarity with screen comes handy on remote Unix shells or systems which don't have virtual consoles yet (such as The Hurd).


7. Using Debian GNU

7.1. Common keystrokes

  • Use Shift + PageUp/PageDown keys to scroll the text that ran out of the visible screen area

  • Ctrl+l to clear the screen (or type clear)

  • Ctrl+s to stop terminal output (actually, to stop the application producing it), and Ctrl+q to resume.

  • You can use splitvt utility to split virtual console in two separate terminals. Once you start it (simply splitvt), you can switch between open consoles with Ctrl+w. Additional keystrokes are:

    • Ctrl+o,q : quit

    • Ctrl+o,4- : make current window 4 lines smaller

    • Ctrl+o,2+ : make current window 2 lines bigger

  • To dump contents of a screen to a file, use screendump command.


7.2. Terminal settings

For more on terminal settings, see the: setterm(1), reset(1), tput(1), tset(1), stty(1), termcap(3) and terminfo(3X) man pages. We'll cover them only briefly here:

  • disable screen blanking (otherwise primitive screensaver)
    setterm -blank 0

  • disable beeps
    setterm -blength 0

    NotePlease Note:
     

    Under X, use xset b off.

  • set foreground and background
    setterm -foreground black; setterm -background white

  • hide/show cursor
    setterm -cursor off; setterm -cursor on
    tput civis; tput cnorm

Also, check out the console_codes(4) man page.


7.3. Learning and using the vim editor

The Unix core is text based and your primary input device should be a keyboard. I can understand that you most probably won't be attracted to an open source OS until you see the Gnome or KDE graphical desktop environments, but when you decide to get deeper in the system, you'll value your keyboard.

Having a text editor (not the word processor, note the difference!) of preference is a must with Unix; almost everything you do will somehow be related to a plain text format.

Most people would tell you to start with joe, nano, pico or some graphical editor ("just until you get the grip"), but I consider this approach to be wrong. Text editing is too important a task to be learned the wrong way from the beginning. After all, there's nothing to be learned about those editors - they are simple, functionality-limited, and their common keystrokes are listed at the bottom of the screen so you should understand the whole story the first time you start them.

The category of professional text editors is reserved for the two old rivals - Richard Stallman's GNU Emacs and Bram Moolenaar's Visual IMproved (or the old traditional vi). As you should have guessed from the title, I prefer vim, but there's no doubt both editors are professional, and one's personal preference should be respected.

I don't want to harass verified Emacs users, they have my permission to skip the rest of this subsection.

VIM is the professional programmers' text editor. Apart from having ultra fast keystrokes, macros, abbreviations, editing modes, syntax highlighting and keyword completion, it can literally solve it's way out of a maze (see /usr/share/doc/vim/macros/maze/).

To install vim and the related tools, you would typically do:

# apt-get install vim vim-gtk vim-scripts exuberant-ctags

Maybe the biggest advantage of vim (or plain old vi in this context) is that it is probably installed on every Unix system you can think of. If you ever get to using some of the traditional Unix operating systems, you'll thank me for it :)

Additionally, you can use vim as the default text editor inside your Gnome apps, using the Bonobo system (and there is a standalone, graphical interface to vim - gvim). KDE users should see kvim.

A very important thing to note is that vim supports a few different editing modes. When you run it, it automatically enters the command mode, meaning that you can't just start typing; letters would be interpreted as commands, not the characters you'd like to appear in the text document.

The most trivial thing you could do is to type "a" or "i" (Append or Insert) when you want to enter the insert mode (so that you can actually type something in your document). To save and quit, you'd press ESC (which switches back to command mode) and type ":wq" (to Write and Quit).

The subject is, of course, too complex to cover here; I'll point you to "vimtutor" and few external resources:

NotePlease Note:
 

Definitely don't miss the opportunity to learn vim or GNU Emacs before continuing! It is an effort that pays off before you even start ;-)


7.4. The readline key and function bindings

Fire up the bash(1) man page, press /^SHELL BUILTIN COMMANDS (to search for that phrase at the beginning of a line), and then search for 'bind' (type /bind).

More information can be found in the /usr/share/doc/bash/ directory.


7.5. User configuration files

As you probably know by now, standard Unix system users do not have write permissions everywhere, the list is often limited to various temporary locations and their home directory.

While most system services keep their configuration files in the /etc/ directory, it's obvious the users can't do the same, they need private and isolated areas to save their preferences (and finally, they don't have permissions to put anything in /etc/). Therefore, their configuration information is saved in dotfiles in their home directory. The dotfile is simply a file whose name starts with a dot (say, ".vimrc"); such files are considered hidden, and are not shown in the directory listing output, unless you use the -a (--all) switch.

Applications generally use fallbacks to load the configuration information. It means they first check the [runtime] command line options which have the highest priority. Next are the environment varibles, followed by dotfiles and finally the generic configs (could be from /etc/).

Typically, an application will create its dotfiles when you run it for the first time.


7.6. Command aliases

Let's say you like the structured output of 'ls -al', but you would like to type 'll' instead of the complete 'ls -al'. The solution to this problem is called 'an alias' and is implemented on the shell level. Edit your ~/.bash_profile and add something like this:

alias ll='ls -al'

Re-read that config file:

# Type
$ source ~/.bash_profile

$ . ~/.bash_profile 

 Ctrl+x,Ctrl+r 

Type ll and enjoy :)

NotePlease Note:
 

As you might have noticed, format of .bash_profile and .bashrc files is very simple, you put the same commands you would type on the command line there. We have saved the alias information in ~/.bash_profile because we want to have it next time we log in.


7.7. Advanced command line features

Most probably, you are using the bash shell (echo $SHELL) with the 'readline' support included. Readline is "the library that handles reading input when using an interactive shell". While traditional Unix users despise bash (a waste of memory, they say), bash and readline form a deadly efficient user environment.

Let's take a look at just few of their features:

  • Type 'history' to see a list of previously executed commands. History is saved in your ~/.bash_history file. New commands get appended to the file after you close a terminal session. If you want to exit without saving history, you can close the shell with kill -9 $$ or kill -9 0 . However, that is not the regular way - the regular way is to read about history feature in the 'bash' manpage and disable it properly (the 'bash' manpage is large, when you open it, type '/hist' to search for sections talking about the history feature).

  • Use Arrow Up and Arrow Down keys to scroll through the history buffer (which is saved between sessions). Use Home and End keys to move cursor position within the line. If they don't work (the Home or End keys), you can achieve the same with Ctrl+a, Ctrl+e (and backspace with Ctrl+h).

  • Use TAB to complete commands or file names. Type 'cat /etc/syslog.' and press TAB, you'll see what it does.

  • Use Alt+. (dot) or Esc,Shift+_ to repeat the last argument from the previous command. To choose which argument you want repeated, use the longer form: Esc, arg_number, Ctrl+Alt+y. Try something like:

    mkdir x
    cd Alt+.

  • Events

    • !! : repeats last command

    • !-1 : repeats last command

    • !co : repeats last command starting with 'co'

    • !n : executes command N from the history list

  • Press Ctrl+R, then start typing a command until you see the line you want (this "reverse search" searches the history buffer for command which is most similar to the fragment of text you type in).

  • When changing directories, there are a few hints: you can get back to previous dir you were in by typing 'cd -'. You achieve the same (although you can put more directory names in the list) with pushd and popd (say, 'cd /bin; pushd /etc/; popd')

  • You would then most probably like to find out the names of programs you can start from the command line. Here's one nice little trick: Type 'a' and press the TAB key twice. It will show you all available commands starting with 'a' (but that's not magic, you can see the same information by just typing 'ls' in /bin, /usr/bin and /usr/local/bin directories).


7.8. Customizing the X session

Just as the name suggests, the file to look for is ~/.xsession. You can find a proper example in /usr/share/doc/xfree86-common/examples/.

The ~/.Xdefaults file would be a place to tune Xresources information (invoke xrdb -load ~/.Xdefaults command from the ~/.xsession file to load it up).


7.8.1. Choosing X backgrounds

Tastes vary; some prefer single-color backgrounds, some like gradients, some set pictures as backgrounds.

I've found it's possible to create very nice-looking backgrounds using the xplanet package (install xplanet and xplanet-images):

$ xplanet -blend -grid  -label -projection mercator -quality 100 -rotate 45   -cloud_image clouds_2000.jpg&
$ xplanet -blend -grid  -label -projection mercator -quality 100              -cloud_image clouds_2000.jpg&
$ xplanet -blend -grid  -label -projection mercator -quality 100 -latitude 15 -cloud_image clouds_2000.jpg&

NotePlease Note:
 

The clouds_2000.jpg picture is located at the xplanet project page and is a realistic map of clouds surrounding the Earth (refreshed every 3 hours, and the image is about 500kb big).

It is possible to have xplanet generate a new picture every 5 minutes; see the xplanetbg(1) manpage and add it to the ~/.xsession file if you like the idea.

Good background pictures can be found in the debian-propaganda package.

Manually, you can set backgrounds with the xsetbg, xsetroot or Esetroot (from the eterm package) commands.


7.8.2. Taking screenshots

# Use the traditional xwd
$ xwd -root >screenshot.xwd; convert screenshot.xwd screenshot.png; pngcrush screenshot.png; rm screenshot.xwd

$ scrot scrnshot.png

$ import -comment "Gnome2 + theme xy" -compress jpeg -frame -screen screenshot.jpg


7.9. Reporting bugs

  • Make sure the bug is reproducible and that you are using the latest version.

  • Set DEBMAIL and DEBFULLNAME environment variables:

    # export DEBMAIL="someone@something.org"
    >

  • # reportbug  -bx [package]

CautionCaution
 

Do not get involved in reporting a bug unless you are 100% sure you have found a real problem; before thinking "It's a bug", ask yourself if you're actually experienced enough (in a particular subject) to say what's a bug and what's not. Then definitely check if the problem was already reported. Package maintainers have enough work to do without your false and ridiculous "bug" reports.

However great it might seem to report bugs (and feel you're "contributing" to the project), you should watch not to be too passionate about it, there are better ways to contribute to Debian (writing documentation and man pages, for example).


8. Debian GNU kernels

8.1. Basic kernel information

A kernel is the essential part of a computer operating system, the core that provides functionality for all other software. It basically consists of low-level services (interrupt handler, process scheduler ...), hardware and pseudo drivers, and a set of system calls which make kernel features available to the operating system and other, higher-level applications.

Every operating system has a kernel, but some of them are distributed under proprietary licenses and don't have the source publicly available so we'll leave them out of the scope.

Probably the most popular free kernel today is Linux, and many so-called distributions have evolved around it. It is very important to note that Debian GNU is not a typical Linux distribution because it is not tied to the Linux kernel exclusively; other Debian ports include GNU/Hurd, GNU/NetBSD and GNU/FreeBSD.


8.2. Kernel recompilation

If you need to recompile your kernel (be it for your personal amusement or a real purpose) - read on; otherwise skip this chapter and just scroll below to see how to install precompiled kernels.

Whether kernel recompilation gives you any real-world benefits is always open for discussion, especially if we're judging between recompilation and the use of a precompiled package which has the optimizations for your processor type (AMD K7 for example), but it can't hurt and from my experience, it is always a good idea. Apart from resulting in a more optimized kernel image and usually much better disk I/O performance, it serves as a kind of a small stress-test for your machine.

NotePlease Note:
 

To test the I/O performance of your disk, do the following (obviously, you'd try it before, and after you boot into the new kernel):

# apt-get install hdparm
>

Additionally, you get the kernel tree with the .config, all the *.o files and configured modversions, which helps you later build additional modules and kernel components that directly fit in the running kernel, without the need for a complete recompilation or reboot.


8.2.1. Linux

First, make sure you have all the necessary ingredients:

# apt-get install gcc make patch bin86 kernel-package libncurses5-dev libc6-dev

The bin86 package is only needed on the ia32 (Intel and compatibles) architecture, and libc6-dev is not critical here, but you'll almost surely need it later.

Then search apt's database for available kernel source versions and install the appropriate one (the highest usually, unless you have some specific needs). The kernel source will be placed in a compressed archive (/usr/src/kernel-source-<version>.tar.bz2), so after the installation we need to additionaly extract the archive, and create /usr/src/linux (by convention, a symbolic link pointing to the kernel source):

$ apt-cache search kernel-source-
>

NotePlease Note:
 

It often happens that there are newer Linux kernel versions released, but they don't appear to be available as Debian packages (at a moment). There's nothing wrong with it; don't complain and just use whatever the highest version available in Debian is. The Debian people responsible for the packages definitely are doing their job very well, so if a particular version is not available, it's either because it needs more testing prior to inclusion and widespread use, or primarily contains backported features and structure - something you don't want to know about 90% of the time.

To check current Linux kernel versions available, you can use the traditional finger tool (which somehow got out of mainstream use, partly due to a rich set of security problems in finger daemons, partly because of global ignorance):

# apt-get install finger
$ finger @kernel.org

If you want to include additional kernel modules in the build (provided their source is available as a Debian package, as it is for 'i2c' and 'lm-sensors' in our example), simply install their packages, uncompress them (they will properly extract into /usr/src/modules/) and move on:

# apt-get install i2c-source lm-sensors-source
>

To quickly discover all additional module sources that are available, you'd do something like this (gives you pretty exact results):

$ apt-cache search -- -source | grep "source "

Non-standard modules which do not have the source available in an existing Debian package are not handled during this stage, and I cover them in the next section.

At this step, you need to pre-configure the new kernel. To do so, run make menuconfig and select your options through a very user-friendly ncurses interface (that's why you installed libncurses5-dev).

New users find this step somewhat problematic because they're presented with literally hundreds of options, and not all of them have an obvious purpose or good documentation available. It should suffice to say that the precompiled package from your repository (kernel-image-2.4.18 in our case) contains the file /boot/config-2.4.18, which is practically a copy of the options the default Debian kernel was built with. So, getting that file and copying it to /usr/src/linux/.config should help you move forward:

# apt-get --download-only kernel-image-2.4.18
>

Optionally, you could still run make menuconfig and exclude the options and drivers you know you won't be needing, just to reduce the overall compile time (however, it usually takes less than 15 minutes on ~1 GHz PCs). Actually, you will want to run it to change the processor type; that option is found somewhere at the top and it is very worth setting to the type that most closely describes your hardware.

Excellent. We now only need to specify an EXTRAVERSION field for our kernel (a string added to the kernel name and file paths, we'll use doc1.0 in this example), and we can start the compilation:

# cd /usr/src/linux
>

If everything goes smoothly (and it should!), you should see some new .deb files generated in the /usr/src/ directory. The Kernel Has Been Compiled :)

Should you recompile the kernel again (after changing options or something), make sure to raise the revision number (doc1.0); you can't create two packages having the same revision (this has to do with the GNU ChangeLog file).

NotePlease Note:
 

We have used the official Debian way to compile the kernel here (make-kpkg). However, the generic kernel compilation guide (found in /usr/src/linux/Documentation/) suggest a different, manual method (make dep; make clean; make bzImage...) and indeed, some people insist on using this "bare bones" approach. I can understand their motives, some exposure to RedHat or SuSE Linux makes you run for the border at any mention of "distribution-specific ways", but I assure you this is not the case here. make-kpkg is a powefull tool that does not violate these healthy principles and, at the same time, it does a lot of things you would miss doing manually (setting up initrd and System.map for example). So, stick to Debian GNU, and stick to its superior techniques for fun and profit.


8.3. Kernel image installation

After the kernel installation and a reboot, use uname -a just to make sure you really are using the new kernel, and optionally configure kernel modules using the modconf tool.


8.3.1. Manually generated packages

8.3.1.1. Linux

If you recompiled your kernel, you should see one or more (depending on any additional modules you compiled in) .deb files in the /usr/src/ directory. You can either copy them to the appropriate place in your local apt package repository (and install with apt-get, just like everything else), or install them all directly with the dpkg command.

# cd /usr/src/
>

In case you want to add a non-standard module (or nVidia proprietary drivers for example, which I prefer to build manually although the nvidia-kernel-source package exists), you should do that later, after you compile, install and boot into your new kernel. Usually, non-standard modules you build from source only require you to have the appropriate kernel running and its corresponding source in /usr/src/linux/, and they will build cleanly. You can then copy the resulting module file(s) somewhere under /lib/modules/`uname -r`, and run depmod -a. Needless to say, you should never run make clean or make distclean inside your kernel tree (unless you plan to build everything again), because the former would cause problems with modversions, and the latter would delete your /usr/src/linux/.config file so new modules wouldn't be able to match up with the existing kernel.


8.3.2. Precompiled Debian packages

8.3.2.1. Linux

In case you did not recompile the kernel yourself, and just want to use some of the existing Debian kernel packages, you can do this:

As usual, first find out the available kernel versions (optimized for AMD's k7 processor series):

$ apt-cache search kernel-image- |  grep k7

Then notice how every kernel version is split into multiple .deb files:

$ apt-cache search 2.4.18

(You of course only need kernel-image-2.4.18-k7 for the start). Simply install it:

# apt-get install kernel-image-2.4.18

8.4. System bootloader

By default, Debian uses the lilo bootloader, which surely does its job, but switching to GRUB (the GRand Unified Bootloader) is always a good idea; GRUB is technically superior and gives so much more freedom. I suppose it didn't catch on so successfully (yet) because it lacked the straightforward installation documentation. Fortunately, you now have very useful and precise notes in /usr/share/doc/grub/README.Debian, and examples are available in the same directory.

For even more advanced bootloader setups (including boot schedules etc...), see the excellent Smart BootManager or Gag.


9. Try to do it yourself first

We will summarize the key points I tried to promote throughout the document:

  • Debian does not underestimate the intelligence of an average system user.

  • It is normal not to know all the usage details directly from your memory (especially if you haven't learned them yet :). Having the correct logic is, however, mandatory to know how to help yourself in Unix.


9.1. A generic tasklist

Here's the theoretical list of tasks you should perform for each problem to get in business:

  • Properly define the problem:

    • What do you want to do?

      Divide your problem into logical steps. If you can't precisely express what exactly do you need done, let it mature overnight. Approach it from as broad a perspective as possible.

    • Does your idea make sense?

      If it works "most of the time", has obvious problems or needs additional effort to cover "special cases", chances are that you got it all wrong; you are using an environment designed to be free of incomplete solutions, remember? Re-evaluate your problem and call again tomorrow.

  • How are problems similar to yours usually solved in Unix/GNU/Debian/Linux?

    Having little experience and trying hard to reinvent the wheel the wrong way is a waste of time and money. Try to find a similar problem and re-use the resolution mechanism.

    • Try searching the man and info pages on your system. Make sure your man-db is installed setuid root (dpkg-reconfigure man-db), then run mandb to re-generate the cache indexes. This will allow you to use the apropos and whatis commands, which can help you find the page(s) containing an answer.

      To extract man or info pages into text files, just take advantage of Unix pipes:
      $ man dpkg | col -b > /tmp/dpkg.man.txt

    • Check out the appropriate directories under /usr/share/doc/. You can use the find and grep utilities to filter the useful material.

    • Check out the relevant web sites; debian.org, The Linux Documentation project or others more closely related to your problem (listed in the Links section of this guide).

    • Use Google, google.com/{linux,bsd}, or the Google groups service and search by keywords (results from the mailing lists will be included too).


9.2. getting help on IRC

IRC, the Internet Relay Chat service, is a very convenient way to get help directly from the community, from the people who use or develop the same software you're using.


9.2.1. IRC servers and channels

  • irc.oftc.net : #kernelnewbies, #offtopic, #lartc, #holarse

  • irc.openprojects.net : #debian, #debian.de, #debian-bugs, #debian-kde, #debian-oo, #c, #perl, #crystalspace, #blenderchat, #vim, #devtools, #prelude, #interchange, #hprog

You can use /msg chanserv info #[channel] to get more information about a specific channel.


9.3. How not to ask questions on IRC

IRC itself is just a protocol. Many written and spoken conventions have been established on top of it (especially since the Free Software networks started appearing).

The rules are very 'natural' and do not impose restrictions, they just define a few things you could do to avoid annoying other people on the channel, and to raise the probability of getting an answer to your question.

Many newcomers start discussing completely off-topic issues (including, but not limited to, their private lives) on channels like #debian (which is a very much Debian-related (who would guess?) place). When someone warns them, they feel insulted and not welcome. Please note that #debian is a big channel (500+ people) and we all have to follow some rules, if we want the whole thing to function.

I've tried to compile a summary (mostly taken from the apt's database [apt is an IRC robot (bot) on #debian]) of things you should[n't] do:

  • If you have a question, please just *ask it*. And don't address specific people - ask the channel. If somebody knows, they'll answer :) To get help quicker, don't ask "can I ask a question," "can anyone help," or "does anybody use/know about foo". Be as specific as you can. Specific, Informative, Complete, Concise, and On-topic. Don't be demanding or insulting; please don't /msg people without permission.

  • Do not start by saying your program doesn't work. Look buddy, doesn't work is an ambiguous statement. Does it sit on the couch all day? Does it want more money? Does it waste its time on IRC all day long? Please be specific! Define 'it' and what it isn't doing.

  • Do not ask questions such as "Can someone guide me through xyz?" or "Could somebody tell me about xyz?". If you're a normal person at all, you'd come to the conclusion that we are not willing to write a book for you (in realtime!) or be your handholder. Most of the subjects we're talking about are very broad, so in absence of your proper problem description we wouldn't even know where to start.

    All the documentation you need is already available, and there's no excuse for not reading it. Asking the channel instead of examining it first (and trying it on your own, of course) is a dangerous shortcut you should not be taking.

  • Don't repeat yourself, use excessive punctuation, paste blocks in non-#flood channels, or repeatedly annoy people in private. "Playing" ascii graphics is the other nasty thing you should avoid.

  • If you're a Debian newbie (a beginner), do not try to answer other people's questions unless you're 100% sure you're right. The reason you came to the channel is learning, not misinforming other people; don't make the channel's life harder.

  • When talking to a specific person in the channel, please prefix the messages with his/her nick. Most IRC clients support TAB nick completion nowadays.

The spectre of possible problems is very broad and the effort needed to describe the problem varies. Generally:

  • Do not press ENTER instead of the Spacebar; do not waste lines. Ideally, everything should fit in a single line (and if it doesn't, your client will probably split it automatically, so you just don't bother about the line length).

  • We assume you tried to solve the problem on your own before asking on IRC, therefore include the information on:

    • How did you obtain the software (i.e. "apt-get install xxx")

    • What was the initial goal you wanted to accomplish

    • How did you approach it

    • Where specifically did you run into trouble

    By constructing such correct and precise questions, you allow people to identify your problem and answer without asking numerous other subquestions.

    Besides, some people try to do things the wrong way. They start asking about an issue, and after 30 mins of discussion, they finally mention it's a part of "something else" they wanted to do, and then we come to the conclusion they took the completely wrong path. By describing the problem properly, you allow other people to evaluate your whole idea, not just the specific problem.


9.4. Frequently used terms

9.4.1. Dpkg, apt, dselect, tasksel

dpkg

dpkg is a medium-level tool to install, build, remove and manage Debian GNU/Linux packages. dpkg itself is controlled entirely via command line parameters, which consist of exactly one action and zero or more options. The action-parameter tells dpkg what to do and options control the behavior of the action in some way.

apt

is a management system for software packages and includes apt-get, apt-cache and apt-cdrom tools. apt-get's strengths are smooth software upgrades, automatic dependency calculation and general convenience.

dselect

is a ncurses (text) frontend to dpkg. It may have more features than apt-get, but requires some time to get used to it (and it's not always as efficient as apt-get would be).

tasksel

allows you to select whole program categories at once (such as, “install games”).


9.4.2. Manual and info pages

man

(as in manual) pages offer uniform interface to documentation under Unix. For example, if the program is called mkdir and I mention it somewhere but you don't know what does it do, type 'man mkdir' and find out.

info

is GNU's way to document things. Some of the serious GNU tools have only short man pages, while the real documentation is hidden in info pages.


9.4.3. System administrator, superuser, root

Terms all referring to the administrative account or an official person which has the privileges to do anything he likes (yes, that includes deleting all your files, locking you out of the system and reporting your real earnings to the IRS :). Don't confuse with 'root directory' below.


9.4.4. Home directory

When you successfully authenticate with the system, you get logged-in, and you enter your 'home directory' under which you have full control to read, write and delete files (you don't have such permissions on the rest of the system). To see which directory it is, type 'pwd' (it could look something like /home/your_name). Wherever you are, type just 'cd' to get back to your home dir. The 'tilde' (~) is a special character that expands to your home directory (try 'echo ~' or 'vim ~/.bashrc').


9.4.5. Root directory

In Unix, disk partitions are visible (we say they are "mounted") in normal directories, and the base directory is always "/", called the "root directory". So for example, in Windows you would have disks like C: and D: but in Unix (and Linux), you connect them to directories, so your D: disk might be mounted to /mnt/windisk-D directory under Linux (the location is arbitrary, of course).


9.4.6. /etc

The /etc directory contains the configuration files for your system. If someone tells you to edit the syslog's config, it means you should go to the /etc/ directory and edit the syslog.conf file. As a general rule, most packages have a config file which is named <package>.conf, or store their config files in /etc/<package>/ directory.

NotePlease Note:
 

For a complete description of the filesystem layout and the system directories you see, check out the Files structure subsection from the Debian User Reference Manual.


9.4.7. /proc

This is a virtual directory which looks like all the others on your system, but its dynamic content is generated directly by the kernel, and not from files on your disk. It is provided as a convenience in kernel-user communication (in both directions, although it was designed read-only in its early stages), and holds kernel and process-related data. Writable files are used to tune system behavior.


9.4.8. Binary

Binary is a common name for any file or data in a binary (two-digit) format. It is often a synonym for an executable file. I could say something like "where's the ttysnoop binary?" and the answer would be "the binary is located in /usr/sbin/ directory".


9.4.9. Image

This term mostly does not refer to a picture or drawing, but to a raw, low-level data. For example, the data cdrom contains the ISO9660 filesystem and some files on it. You could use the traditional Unix dd utility to create an image of that cdrom on the hard disk (the output would, of course, be a single raw file, a direct copy of bits from the original media); it also means that the file would look like a valid device and you could even mount it locally (using the 'loop' option).


9.4.10. Daemon

A system process which runs in background, normally for a long period of time, and is dedicated to servicing user requests. Examples of daemons are httpd (Apache), telnetd, ftpd (ProFtpd) or sshd (Openssh).


9.4.11. X11R6, XFree86

X11R6 (the "XWindow System") is an industry standard for windowing applications. XFree86 is an open-source implementation of the X11 protocol used by most free operating systems today (see XFree86 website).


9.4.12. RMS

Richard Matthew Stallman (see picture), ace MIT hacker, founder of the Free Software Foundation.


9.4.13. curses

The curses (or the new, ncurses) library is a set of routines which gives the user a terminal-independent method of updating character screens with reasonable optimization. Linux uses the ``new curses'' (ncurses) implementation and it is the approved replacement for 4.4BSD classic curses, which has been discontinued.

So, when we say apt-setup is a ncurses application, we mean it has a nice textual user interface with windows, menus and buttons. Ncurses applications are generally very convenient and nice to use. I've found a link to the ncurses programming tutorial.


9.4.14. The Unix Epoch

The time and date corresponding to 0 in an operating system's clock and timestamp values. Under most Unix versions the epoch is 00:00:00 GMT, January 1, 1970; under VMS, it's 00:00:00 of November 17, 1858 (base date of the U.S. Naval Observatory's ephemerides); on a Macintosh, it's the midnight beginning January 1 1904.

System time is measured in seconds or ticks past The Epoch.

Weird problems may ensue when the clock wraps around, which is not necessarily a rare event; on systems counting 10 ticks per second, a signed 32-bit count of ticks is good only for 6.8 years. The 1-tick-per-second clock of Unix is good only until January 18, 2038, assuming at least some software continues to consider it signed and that word lengths don't increase by then.


2003-08-07 00:27:26